NBenchmark 0.46.0

NBenchmark

Straightforward benchmarking for .NET.

Benchmarking sounds simple - run it, time it, compare. In practice the numbers are easy to get wrong: the JIT is still optimizing your method during the first runs, the timer can cost more than a fast method, one GC pause skews an average, and the 2% improvement you were sure you measured can be noise.

NBenchmark handles that for you. One line gives you a calibrated, warmed-up, outlier-trimmed result with a confidence interval.

var result = Benchmark.Run(() => MandelbrotCalculation());
result.Print();

Why NBenchmark?

• No setup. One static call. No attributes, no class structure, no dedicated project.
• No numbers to guess. Warmup, batch size, and sample count all resolve themselves.
• Clean process by default. Your numbers reflect your code, not your process's history.
• Real statistics. Confidence intervals and significance testing, not just an average.
• Zero dependencies. The core package is BCL-only.

Installation

dotnet add package NBenchmark

Optional packages

Four modes, one engine

1. Single mode

The fastest way to get a reliable number.

var result = Benchmark.Run(() => int.Parse("12345"));
Console.WriteLine($"P95: {result.GetPercentile(0.95)} ns, Alloc: {result.MeanAllocatedBytes} B");

var result = await Benchmark.RunAsync(async () => await FetchDataAsync());

2. Suite mode

Compare multiple implementations with a fluent API.

var results = await new BenchmarkSuite("string concat")
    .Add("plus operator", () => "a" + "b" + "c")
    .Add("interpolation",  () => $"{"a"}{"b"}{"c"}")
    .WithBaseline("plus operator")
    .WithReporter(new ConsoleReporter())
    .RunAsync();

The output includes a Ratio column and a ✓ in the Sig column when the speed difference is statistically significant.

3. Harness mode

Attribute-based discovery with a CLI, for dedicated benchmark projects.

public class StringBenchmarks
{
    [Benchmark(Baseline = true)]
    public string Concat() => "a" + "b" + "c";

    [Benchmark]
    public string Interpolate() => $"{"a"}{"b"}{"c"}";
}

await BenchmarkHarness.Create(args)
    .AddFromAssembly<StringBenchmarks>()
    .WithReporter(new ConsoleReporter())
    .RunAsync();

dotnet run -- --filter StringBenchmarks.Concat  # Run a specific benchmark
dotnet run -- --dry-run                         # Validate wiring without running
dotnet run -- --reporter json                   # Output results for CI/CD

4. Global tool

Install once, benchmark any assembly with [Benchmark] methods - no project needed.

dotnet tool install -g NBenchmark.Tool
dotnet benchmark --project ./MyApp.Benchmarks
dotnet benchmark --assembly ./bin/Release/net10.0/MyLib.dll

All harness CLI flags pass through (--filter, --reporter, --output, --threshold-pct, etc.).

Features

Built on real statistics

• Adaptive measurement. Samples stream until the confidence interval is tight enough, then stop. Warmup ends when the timings plateau and the JIT has settled - not after a guessed count. (Measurement)
• Error bars that survive trimming. Discarding an outlier does not narrow the confidence interval: a discarded sample still counts as an observation, so the reported margin describes the run that happened rather than the samples that survived it. (Outlier trimming)
• Non-parametric significance testing. Benchmark timings are not normally distributed, so the built-in tests are rank-based. A ✓ in the Sig column means "real, and at least a small effect", not merely p < 0.05. (Significance testing)
• Verified against SciPy and NumPy. Every statistical primitive is dependency-free and cross-validated on each build. (Validation)

View the full documentation at docs.nbenchmark.net.