Bitzsoft.Integrations.AI.Testing 1.2.0

dotnet add package Bitzsoft.Integrations.AI.Testing --version 1.2.0
                    
NuGet\Install-Package Bitzsoft.Integrations.AI.Testing -Version 1.2.0
                    
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Bitzsoft.Integrations.AI.Testing" Version="1.2.0" />
                    
For projects that support PackageReference, copy this XML node into the project file to reference the package.
<PackageVersion Include="Bitzsoft.Integrations.AI.Testing" Version="1.2.0" />
                    
Directory.Packages.props
<PackageReference Include="Bitzsoft.Integrations.AI.Testing" />
                    
Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.
paket add Bitzsoft.Integrations.AI.Testing --version 1.2.0
                    
#r "nuget: Bitzsoft.Integrations.AI.Testing, 1.2.0"
                    
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
#:package Bitzsoft.Integrations.AI.Testing@1.2.0
                    
#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package.
#addin nuget:?package=Bitzsoft.Integrations.AI.Testing&version=1.2.0
                    
Install as a Cake Addin
#tool nuget:?package=Bitzsoft.Integrations.AI.Testing&version=1.2.0
                    
Install as a Cake Tool

Bitzsoft.Integrations.AI.Testing

面向 AI、Agent 与 MCP 连接器下游宿主的独立测试支持包。聚合全部无测试框架锁定(不传递引入 xUnit / NUnit / MSTest)的存储端口契约测试基类、进程内测试替身、受控时钟以及基于 Microsoft.Extensions.AI.Evaluation 的 FDE 黄金数据集评估运行器。

安装与兼容性

dotnet add package Bitzsoft.Integrations.AI.Testing
  • 目标框架:net8.0;net10.0(全面开启 <IsAotCompatible>true</IsAotCompatible>,零反射序列化);
  • 1.2.0 迁移与破坏性变更说明:自 1.2.0 起 MCP 契约测试基类(McpAccessTokenStoreContractTestsBase、McpCandidateStoreContractTestsBase)依赖已由 Bitzsoft.Integrations.McpServer 切换至 Bitzsoft.Integrations.AI.Mcp。测试项目升级本包至 1.2.0 时请同步将旧包 McpServer 原子切换为 AI.Mcp 并更新 using 命名空间(详见 ADR-0015);
  • 仅面向宿主的 *.Tests 测试项目安装,严禁被生产运行时程序集引用;
  • 核心生产包(AI.Governance、AI.Conversation 等)对本包保持零依赖。

契约测试基类列表(共 23 个)

基类 验证端口 / 运行期能力 核心不变量
AIPendingActionStoreContractTestsBase IAIPendingActionStore 跨租户隔离、会话去重、乐观并发抢占、到期扫描、取消传播
AIUsageCounterStoreContractTestsBase IAIUsageCounterStore 固定窗口累加、过期归零、键隔离、高并发增量、取消传播
AIRequestLedgerStoreContractTestsBase IAIRequestLedgerStore 三元组幂等创建、乐观并发跃迁、心跳续期版本保护、会话租约互斥、取消传播
AIProviderCatalogStoreContractTestsBase IAIProviderCatalogStore 启用过滤与排序、租户与工作区隔离、健康状态幂等写入、取消传播
AISkillDefinitionStoreContractTestsBase IAISkillDefinitionStore 内置基准幂等同步、全局与租户覆盖隔离及删除恢复、取消传播
AIAgentDefinitionStoreContractTestsBase IAIAgentDefinitionStore 内置智能体幂等同步、全局与租户覆盖隔离及删除恢复、取消传播
AIConversationSessionStoreContractTestsBase IAIConversationSessionStore 租户/用户归属隔离、工作区绑定、并发版本保护、消息序号与修补、分叉与回合重放、取消传播
AIBackgroundJobStoreContractTestsBase IAIBackgroundJobStore 幂等建单与租户隔离、优先级抢占与租约到期接管、CAS 跃迁与终态保护、取消传播
McpAccessTokenStoreContractTestsBase IMcpAccessTokenStore 哈希一致签发、受控时钟过期与清理、本人/管理面吊销隔离、取消传播
McpCandidateStoreContractTestsBase IMcpCandidateStore 候选批量/单条更新、黑名单增删查确定性排序、取消传播
MemoryStoreContractTestsBase IAIMemoryStore 租户与用户物理隔离、双区块字符与条数容量硬上限、提示注入与密钥阻断、增改删幂等、取消传播
SkillResourceReaderContractTestsBase IAISkillResourceReader 跨租户隔离、正式态与沙箱草稿态可见性隔离、路径穿越别名阻断、分页与安全围栏闭合、取消传播
ExecutionArtifactStoreContractTestsBase IExecutionArtifactStore 三重隔离、分页切片、受控时钟 TTL 过期回收、会话级清理隔离、取消传播
ArtifactAnalysisEngineContractTestsBase IArtifactAnalysisEngine 跨表连接、十进制精度保真、同环比除零保护、派生血缘追踪、跨租户拦截
TurnFeedbackStoreContractTestsBase IAITurnFeedbackStore 租户/用户/请求幂等覆盖、跨租户与跨用户隔离、UTF-16 安全截断、取消传播
SkillDraftStoreContractTestsBase IAISkillDraftStore 沙箱草稿态保存、租户隔离、状态流转与审核隔离、取消传播
ScheduledAgentTriggerStoreContractTestsBase IAIScheduledAgentTriggerStore 租户/用户隔离、到期查询、幂等认领与冷却窗去重、乐观并发版本保护、取消传播
CrossProviderFailoverContractTestsBase OrderedFailoverChatClient / IAIFailoverHealthTracker / IAICrossProviderHistorySanitizer 首片前丢弃脏缓冲切换、首片后绝不重放、跨厂商剥离加密签名与归一化工具调用 ID、租户健康隔离与半开防惊群
ModelRoleIsolationGuardContractTestsBase HermesRoleRoutedChatClient / AIModelExecutionRole / ITurnTriageRouter 轻量角色(门控/压缩/内务/心跳)绝不误触主推理模型、未配置独立轻模型时零开销旁路
ProviderReasoningEffortContractTestsBase AIReasoningEffortPolicy / HermesProviderCapabilityRegistry 自定义网关模型映射、关闭禁发参、单档位拒绝不误伤高阶能力、自动循环上限钳制、取消传播
ResponseVerifierPipelineContractTestsBase AIResponseVerificationHook / AIVerifierOptions / IAgentResponseVerifier 五态核查处置模式、按核查器覆写、单次自纠回环、阻断安全兜底文案与最终裁决动作遥测
ToolSubgraphSelectorContractTestsBase IToolSubgraphSelector / IGlossaryExpander 两跳桥接、词汇衰减展开、RBAC 前置优先红线、环图防死循环与预算硬顶、取消传播
UiComponentCatalogContractTestsBase IAiUiComponentCatalog / IPresentationGuard 五类内置组件预注册、双重载自定义校验、不可绕过安全基线、数据引用多租户隔离、呈现守卫降级阈值与线程安全

测试替身与 FDE 黄金评估运行器

  • AITestFakeTimeProvider:可控 TimeProvider 测试时钟,支持确定性推进租约过期与限流时间窗口;
  • AITestFakeChatClient:可脚本化队列、流式分片、TTFT 时钟快进、工具调用仿真及调用轨迹审计的 IChatClient 替身;
  • AITestFakeCacheEpochCoordinator:支持可用性故障开关、单调纪元计数与 TTL 过期的 IAICacheEpochCoordinator 替身;
  • AITestFakeOutboundRedactor:支持敏感词替换规则、跨境区域阻断策略及调用审计的 IAIOutboundRedactor 替身;
  • AITestInMemoryProviderCatalogStore:内置多租户与工作区过滤及健康事件追踪的 IAIProviderCatalogStore 进程内参考实现;
  • AITrajectoryReplayRunner / AITrajectoryBehavioralAssertions:基于脱敏 JSONL 轨迹执行锁定历史工具返回、零外部副作用的离线确定性重放与工具调用序列/缓存命中率/数字对账行为断言;
  • AIGoldenEvaluationRunner / AIHermesTrajectoryEvaluator / AILlmRubricEvaluator / AIGoldenSuiteRegressionComparator:基于 Microsoft.Extensions.AI.Evaluation 10.11.0 组合 IEvaluator / CompositeEvaluator,覆盖 8 项 hermes.* 指标、零推理快路裁判(解析失败返回 Inconclusive)、FinOps 成本上限、出站脱敏与跨版本退化基线对比(ToMarkdownSummary())。

完整最小示例

下面演示结合 AITestFakeTimeProvider、AITestFakeChatClient、AITestFakeOutboundRedactor 与 AIGoldenEvaluationRunner 执行单条黄金评估验证:

using Bitzsoft.Integrations.AI.Abstractions;
using Bitzsoft.Integrations.AI.Testing;
using Microsoft.Extensions.AI;

var clock = new AITestFakeTimeProvider(new DateTimeOffset(2026, 11, 14, 0, 0, 0, TimeSpan.Zero));
var chatClient = new AITestFakeChatClient(clock)
    .EnqueueTextResponse(
        "已完成季度结算对账单关键指标合规核查。",
        usage: new UsageDetails { InputTokenCount = 1_000, CachedInputTokenCount = 400, OutputTokenCount = 200 },
        toolCalls: [new FunctionCallContent("call-1", "query_settlement_items")],
        timeToFirstToken: TimeSpan.FromMilliseconds(120),
        interTokenDelay: TimeSpan.FromMilliseconds(30));

var redactor = new AITestFakeOutboundRedactor()
    .AddSensitiveTerm("440301199001011234", "[ID_REDACTED]");

var runner = new AIGoldenEvaluationRunner(new AIGoldenEvaluationOptions
{
    TimeProvider = clock,
    OutboundRedactor = redactor,
    DefaultPricing = new AIFinOpsModelPricing(InputPricePerMillion: 2.0m, OutputPricePerMillion: 8.0m, CachedInputPricePerMillion: 0.5m),
});

AIGoldenCaseReport report = await runner.EvaluateCaseAsync(
    chatClient,
    new AIGoldenEvaluationCase
    {
        CaseId = "settlement-review-001",
        Messages = [new ChatMessage(ChatRole.User, "请核验客户 440301199001011234 的季度结算对账单关键指标。")],
        RequiredResponseSubstrings = ["关键指标"],
        ForbiddenLeakedTerms = ["440301199001011234"],
        MaxTimeToFirstTokenMs = 300,
        MaxTotalLatencyMs = 800,
        MaxEstimatedCost = 0.05m,
        ExpectedToolCalls = ["query_settlement_items"],
    });

Console.WriteLine($"Passed={report.Passed}, TTFT={report.TimeToFirstTokenMs}ms, Cost={report.EstimatedCost} {report.CurrencyCode}");
Product Compatible and additional computed target framework versions.
.NET net8.0 is compatible.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed.  net9.0 was computed.  net9.0-android was computed.  net9.0-browser was computed.  net9.0-ios was computed.  net9.0-maccatalyst was computed.  net9.0-macos was computed.  net9.0-tvos was computed.  net9.0-windows was computed.  net10.0 is compatible.  net10.0-android was computed.  net10.0-browser was computed.  net10.0-ios was computed.  net10.0-maccatalyst was computed.  net10.0-macos was computed.  net10.0-tvos was computed.  net10.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last Updated
1.2.0 0 11/21/2026
1.1.0 89 11/14/2026