Cratis.Specifications.XUnit 4.2.0

Cratis Specifications

Specification by Example (BDD) for .NET — Given/When/Then specs with xUnit, NUnit and TUnit, in the style of Machine.Specifications.

Cratis Specifications brings Specification by Example — behavior-driven development (BDD) with given, when, then — to .NET test projects, keeping the concise structure and syntax popularized by Machine.Specifications while running on standard test frameworks. That means full tooling support across IDEs, editors, and CI, with your choice of xUnit, NUnit or TUnit as the runner. The background and motivation are described in this article.

Packages:

• Cratis.Specifications.XUnit — for xUnit v2
• Cratis.Specifications.XUnit.v3 — for xUnit v3
• Cratis.Specifications.NUnit — for NUnit 4
• Cratis.Specifications.NUnit5 — for NUnit 5
• Cratis.Specifications.TUnit

What does it do?

In BDD one talks about the given, when, then. Much like arrange, act and assert in a way that is more common in TDD. The biggest difference is on a mindset level of thinking in specifications of behaviors in your system. What this particular library delivers is a way to do these and also keep in line with what is common in the BDD world of having isolated specifications and not have typically a FooTests and dump all your tests for the unit Foo in it.

The library supports the convention lifecycle methods Establish(), Because() and Destroy(). There is no virtual method to override, just match the expected signatures:

If your specification requires to run in an async context, it also supports the following:

All lifecycle methods are optional and will be ignored if not there. Multiple levels of inheritance recursively is supported, meaning that specifications will run all the lifecycle methods from the lowest level in the hierarchy chain and up the hierarchy (e.g. no_user_authenticated → when_authenticating_a_null_user).

To get all this to work, all you need to do is inherit from the Specification type found in Cratis.Specifications.

Structure and naming

The general purpose of BDD and specification by example is to make it all very human readable and possible to navigate quite easily. New developers can come into the solution and pretty much read up on the specifications and get a glimpse of how the system works. So rather than having a FooTests class with all the tests, it is recommended to have folders describing the scenario being specified. For a unit this could be named for_<name of unit> e.g. : for_SecurityService. If you're testing a more domain centric scenario in your system that involves multiple units, the folder name would reflect the name of the scenario e.g.: for_logging_in_users.

Within these folders you'd keep your when statements. E.g. When_authenticating_an_admin_user. If you want to group things, for instance lets say you have multiple behaviors within the concept of authenticating, you could then have a folder grouping these called When_authenticating and then drop in the behavior specifications within this folder an_admin_user and a_null_user.

In addition to this you might want to reuse a context. This can quite easily be achieved through inheritance. Structure-wise you'd then have a given folder and namespace where you'd put the common reusable context - again reflecting what it represents, for instance for our authentication scenario: no_user_authenticated.

For a sample of how this looks like, look within the sample folder.

Compiler Warnings

Since the naming of classes, methods and structure deviates from what is expected by default from the C# compiler, you typically end up getting a lot of warnings. These can be turned off by adding a NoWarn element within a PropertyGroup to your .csproj file:

<PropertyGroup>
    <NoWarn>CA1707;CS1591;RCS1213;IDE0051;IDE1006;CA1051</NoWarn>
</PropertyGroup>

If you're using static code analysis and stylecop and have turned on all rules by default, you might also encounter the following that you want to turn off:

Depending on your solution, you might want to consider suppressing the following.

Example

In the sample folder you'll find samples of using it XUnit, NUnit and TUnit.

NUnit 4 or NUnit 5

Pick the package that matches the NUnit version your test project uses. Both offer the same API and the same namespace, so switching is only a package change:

One difference: collection.ShouldEqual(1, 2, 3) compares the sequence, including order, in the NUnit 5 package (as in the xUnit and TUnit packages), while the NUnit 4 package has always ignored the order and keeps doing so for compatibility.

The difference between XUnit and NUnit is on the signature of the tests. Instead of [Fact] for XUnit, you'll have to adorn with [Test], in addition to that, NUnit requires methods to be public to run them. TUnit uses [Test] as well, and also requires public methods.

Basically, building on the Machine.Specifications sample - this would become:

class When_authenticating_an_admin_user : Specification
{
    SecurityService subject;
    UserToken user_token;

    void Establish() =>
             subject = new SecurityService();

    void Because() =>
             user_token = subject.Authenticate("username", "password");

    [Fact] void should_indicate_the_users_role() =>
        user_token.Role.ShouldEqual(Roles.Admin);

    [Fact] void should_have_a_unique_session_id() =>
        user_token.SessionId.ShouldNotBeNull();
}

Catching an exception and testing for the correct exception:

class When_authenticating_a_null_user : Specification
{
    SecurityService subject;
    Exception result;

    void Establish() =>
             subject = new SecurityService();

    void Because() =>
             result = Catch.Exception(() => subject.Authenticate(null, null));

    [Fact] void should_throw_user_must_be_specified_exception() =>
        result.ShouldBeOfExactType<UserMustBeSpecified>();
}

Building reusable contexts (in a sub-namespace with given):

class no_user_authenticated
{
    protected SecurityService subject;

    void Establish() =>
             subject = new SecurityService();
}

Refactor one of the specifications:

class When_authenticating_a_null_user : given.no_user_authenticated
{
    Exception result;

    void Because() =>
             result = Catch.Exception(() => subject.Authenticate(null, null));

    [Fact] void should_throw_user_must_be_specified_exception() =>
        result.ShouldBeOfExactType<UserMustBeSpecified>();
}

Supports teardown through destroy:

class no_user_authenticated
{
    protected SecurityService subject;

    void Establish() =>
             subject = new SecurityService();

    void Destroy() => subject.Dispose();

}

Using TUnit

Add the Cratis.Specifications.TUnit package next to the TUnit package in your test project:

dotnet add package TUnit
dotnet add package Cratis.Specifications.TUnit

Specifications derive from Specification exactly as with the other frameworks, and facts are public [Test] methods:

public class when_adding_an_item : Specification
{
    List<string> _items;

    void Establish() => _items = [];
    void Because() => _items.Add("item");

    [Test] public void should_contain_the_item() => _items.ShouldContain("item");
}

The lifecycle matches the xUnit shared mode and NUnit: each specification class is constructed once, Establish and Because run once before the first fact, all facts share that instance, and Destroy runs once after the last fact. If Establish or Because fails, every fact of the class fails. TUnit may run the facts of a class in parallel, so facts must only read from the shared context. Because the instance is shared, do not implement IDisposable on a specification: TUnit would dispose it after every fact. Use Destroy instead.

TUnit assertions are asynchronous, so the Should* extension methods in this package are implemented without them and throw a SpecificationAssertionException with the expected and actual values. They have the same names and signatures as in the xUnit package.

TUnit runs on Microsoft.Testing.Platform. On the .NET 10 SDK, dotnet test for a TUnit project requires opting in to the Microsoft.Testing.Platform runner in global.json ("test": { "runner": "Microsoft.Testing.Platform" }), which applies to every test project in the repository. Use dotnet run --project <spec project> when the repository also has xUnit or NUnit projects on VSTest, as this repository does.

Shared instance per specification class (opt-in)

By default, xUnit constructs a fresh instance of the test class for every [Fact], which means Establish and Because run once per fact rather than once per behavior. For specs whose Because drives real machinery, that multiplies the cost of the action by the number of facts — and a behavior described by several facts ends up being executed several times.

The xUnit package ships an opt-in execution mode with MSpec-style semantics. Add the standard xUnit assembly attribute to your test assembly:

[assembly: Xunit.TestFramework("Cratis.Specifications.SpecificationTestFramework", "Cratis.Specifications.XUnit")]

With this in place, every eligible specification class is constructed once, Establish and Because run once, all its facts run against that shared instance, and Destroy runs once after the last fact.

A class is eligible when it derives from Specification, has a single public parameterless constructor, does not implement IDisposable and contains only plain facts. Everything else — plain xUnit test classes, theories, classes using fixtures or ITestOutputHelper — keeps the stock per-fact lifecycle, so the attribute is safe to add to an existing spec assembly.

The one rule to honor: facts must not mutate the shared context. They are assertions about the single execution the class describes — which is the BDD model to begin with.

xUnit v2 and xUnit v3

Pick the package that matches the xUnit version of your test project:

Both packages expose the same Specification base class, Establish/Because/Destroy conventions and Should* assertion extensions, so specifications move between them unchanged. An xUnit v3 test project is an executable, so set <OutputType>Exe</OutputType> as described in the xUnit v3 migration guide.

The shared instance per specification class is opt-in on xUnit v3 too, with the same eligibility rules. xUnit v3 takes the framework type rather than its name:

[assembly: Xunit.TestFramework(typeof(Cratis.Specifications.SpecificationTestFramework))]

Analyzers

The packages ship Roslyn analyzers (CRSPEC0001–CRSPEC0008) that catch common specification mistakes.

CRSPEC0008 — facts should not modify the shared specification context — reports a should_* fact that assigns to, increments, or calls Add/AddRange/Remove/RemoveAt/Clear/Insert on an instance field or property of the specification. Because it would otherwise break builds that treat warnings as errors, it is reported as info by default. Raise it in an .editorconfig:

[*.cs]
dotnet_diagnostic.CRSPEC0008.severity = warning   # or error

Related projects

Specifications pairs naturally with Synopsis, which turns your specs into living documentation, and it is the testing style used throughout the Cratis stack — including Chronicle, the event-sourcing database and runtime, and Arc, the CQRS application framework for ASP.NET Core.

The Cratis ecosystem

This project is part of Cratis — free, MIT-licensed tools for building event-sourced and CQRS applications.

• Chronicle — event-sourcing database and runtime. Orleans-based kernel, pluggable storage (MongoDB default; PostgreSQL, SQL Server, SQLite, in-memory), language-agnostic gRPC contracts. Docs
• Chronicle clients — first-class .NET SDK, plus TypeScript, Kotlin/Java, and Elixir; Python coming soon (pre-alpha). AI agents connect through the Chronicle MCP server.
• Arc — opinionated CQRS framework for ASP.NET Core with commands, queries, validation, authorization, and TypeScript proxy generation. Works without event sourcing. Docs
• Components — React components aligned with Arc patterns. Docs
• CLI + Workbench — inspect and diagnose Chronicle from the terminal or the browser. Docs
• Model-first layer (experimental) — Studio, Screenplay, Stage, Scene, Prologue
• Supporting — Fundamentals, Specifications (this project), Synopsis, Lens, Narrator, and free AI tooling (preview); Ensemble coming soon (pre-release)
• Samples — runnable event sourcing and CQRS samples for the whole stack

Everything Cratis publishes today is MIT licensed and free to use.

Release notes and announcements: the Cratis blog.