NBenchmark 0.50.0

NBenchmark

Straightforward benchmarking for .NET.

Benchmarking sounds simple: run it, time it, and compare. In practice, you can easily get the numbers wrong. The JIT might still be optimizing your method during the first runs, the timer can cost more than a fast method, a single GC pause can skew an average, and a small improvement might just be noise.

NBenchmark handles these issues for you. One line of code provides a calibrated, warmed-up, and outlier-trimmed result with a confidence interval.

var result = Benchmark.Run(() => MandelbrotCalculation());
result.Print();

Why NBenchmark?

• No setup. Use one static call. You don't need attributes, a specific class structure, or a dedicated project.
• No guessing. Warmup, batch size, and sample count resolve automatically.
• Worker process by default. Results reflect your code rather than the history of the host process.
• Real statistics. Use confidence intervals and significance testing instead of simple averages.
• Zero dependencies. The core package uses only the BCL.

Installation

dotnet add package NBenchmark

Optional packages

Four modes, one engine

Single mode

Single mode is the fastest way to get a reliable number.

var result = Benchmark.Run(() => MyMethod());
var result = await Benchmark.RunAsync(async () => await FetchAsync());

Suite mode

Use suite mode to compare implementations side-by-side with ratios and significance testing.

var results = await new BenchmarkSuite("string concat")
    .Add("plus operator", () => "a" + "b" + "c")
    .Add("interpolation",  () => $"{"a"}{"b"}{"c"}")
    .WithBaseline("plus operator")
    .WithReporter(new ConsoleReporter())
    .RunAsync();

The output includes a Ratio column and a ✓ in the Sig column when the speed difference is statistically significant.

Harness mode

Harness mode provides attribute-based discovery with a built-in CLI for dedicated benchmark projects.

public class StringBenchmarks
{
    [Benchmark(Baseline = true)]
    public string Concat() => "a" + "b" + "c";

    [Benchmark]
    public string Interpolate() => $"{"a"}{"b"}{"c"}";
}

await BenchmarkHarness.Create(args)
    .AddFromAssembly<StringBenchmarks>()
    .WithReporter(new ConsoleReporter())
    .RunAsync();

dotnet run -- --filter StringBenchmarks.Concat  # Run a specific benchmark
dotnet run -- --dry-run                         # Validate wiring without running
dotnet run -- --reporter json                   # Output results for CI/CD

Global tool

Install the global tool once to benchmark any assembly with [Benchmark] methods without needing a project.

dotnet tool install -g NBenchmark.Tool
dotnet benchmark --project ./MyApp.Benchmarks
dotnet benchmark --assembly ./bin/Release/net10.0/MyLib.dll

All harness CLI flags pass through (--filter, --reporter, --output, --max-regression-percent, etc.).

Features

Built on real statistics

NBenchmark doesn't use a simple average of a fixed loop.

• Adaptive measurement. Samples stream until the confidence interval is tight enough and then stop. Warmup ends when the timings plateau and the JIT settles, rather than after a guessed count. (Measurement)
• Error bars that survive trimming. Discarding an outlier doesn't narrow the confidence interval. A discarded sample still counts as an observation, so the reported margin describes the run that occurred rather than only the samples that survived. (Outlier trimming)
• Non-parametric significance testing. Benchmark timings are not normally distributed, so the built-in tests are rank-based. A checkmark (✓) in the Sig column means the effect is real and at least small, not merely that p < 0.05. (Significance testing)
• Cross-validated results. Every statistical primitive is dependency-free and cross-validated on each build against SciPy and NumPy. (Validation)

View the full documentation at docs.nbenchmark.net.