PlayNicely.Executor 1.0.0-prerelease-20240209-122436

Play Nicely Executor

The Play Nicely Executor project supports the execution of MSBuild test case projects within a pre-defined test case environment. Where you can define required commands, or exclude commands from the path (so you can force things to fail). This package is published independently having a solution that supports the creation of a reliable and clean environment, to run tool testing seems like a use case for many scenarios.

The main artefacts are the ITestEnvironment, which defines the environment and project that are available for testing. You create (build) an ITestEnvironment using the fluent interface TestEnvironmentBuilder. With this class you can define what can, and can't run, plus the target test case project. After building your ITestEnvironment you can pass it to a concrete implementation of the ITestEnvironmentRunner which executes the test and returns the outcome.

The current concrete ITestEnvironmentRunners are a basic ProcessRunner which excepts a command and arguments, much like a command line, and returns a basic IExecutionResult. And the DotNetRunner which executes a dotnet subcommand, with arguments, and returns a IExecutionResult<DotNetExecutionContext> which includes the dotnet command results context, targets ran, projects built, lists of errors, etc.

You can also define your own runners derived from ITestEnvironmentRunner.

This project came about to support the use of NodeJS packages within .NET projects in a .NET first way. You can acheive the integration of NodeJS tools using plugins or other tooling. The problem with this is, these can often become out of date or stale. Most of the NodeJS packages are actively developed by a community, so accessing the latest npm packages directly makes the most sense.

Getting Started

This getting started is provided in the context, for which, this package was developed. To support the testing of a NodeJS tools package within a MSBuild project.

The basic setup involves having a tools project, and an accompanying test project, I currently use SpecFlow. Develop your tools and targets in your tools project. The test project includes

Below here is not done.

If you are going to use this package, you probably want to define test cases for a NodeJS tool package. Before you start, it's a good idea to consider the best layout for your project so that you can use a familar development experience for test cases, i.e. Visual Studio, while also making the maintenance of the test case packages as simple as possible.

Our recommended project structure is (if you're using SpecFlow for testing):

solution-dir
|-- my-project	               # This is the project you are developing
|   |-- *
|-- my-project.Specs           # BDD Specifications for your project
|   |-- Features
|   |   |-- *.features
|   |-- TestCases.Projects     # Define your test case 'packages' in resx files
|       |-- Project1.resx
|       |-- Project2.resx
|       |-- TestCases.cs       # Helper class to load local test case projects from embedded resources
|-- TestCase.Projects          # Define your test case projects in here using familiar tools.
|   |-- Project1
|   |   |-- Project1.csproj
|   |-- Project2
|       |-- Project2.csproj
|-- my-solution.sln

We'll refer to this structure in the rest of this document.

Create your test case projects

Use visual studio to create you test case projects, as you would any other C# project. Add them to the solution-dir/TestCase.Projects directory. Add code files, define properties, etc. Typically, you do not want to build your test cases as part of the normal Visual Studio build command. To disable build use the solution Configuration Manager... to disable build on your test cases.

Define test case packages as embedded (resx) resources

For consistency, when running test cases, a clean temporary directory is created for each test run. This means build artefacts like obj or cached NuGet packages are not available, and ensures consistent test behaviour.

So a clean version of the project can be created, they need to be explicitly defined in embedded resource (resx) files.

1. In your BDD project, my-project.Specs, under the TestCase.Projects directory, add a new resx file. It is not required, but is better for consistency, to name your resx the same as your actual project. In the example, above, Project1.resx matches Project1.csproj
2. Disable code generation in your resx file.
3. Add the relevant files from your test case project to your resx.
   1. In the resource editor, Add existing file...
   2. The default naming for added files, mangles the names and removes the extension. As we are trying to define files in a file system like format, you need to re-add the extension and also any directories, using / as a separator.
4. If you have multiple projects in your test case, like you would in a solution, you can specify the target build project by adding a resource string to the resx called ProjectFile with the value being the path (within the context of the resx) to the project file. In the example above, for Project1.resx, the ProjectFile value would be Project1.csproj.

Add a test case helper

This might be simpler than you think, once we bring over the TestEnvironment stuff.

Use steps in your BDD project

This should probably be centralized also.