Corvus.Testing.ReqnRoll

This provides a library of useful testing extensions, primarily focussed on ReqnRoll operations in v4.

It is built for .NET 8.0

The ReqnRoll specific libraries contain additional bindings; to use these, you will need to add a reqnroll.json file to any project wishing to use them. Add entries to the stepAssemblies array for each of the Corvus libraries you need to use:

{
  "$schema": "https://schemas.reqnroll.net/reqnroll-config-latest.json",
  "stepAssemblies": [
    { "assembly": "Corvus.Testing.ReqnRoll" },
    { "assembly": "Corvus.Testing.AzureFunctions.ReqnRoll" },
  ]
}

Features

This library offers the following features.

Container Bindings (Corvus.Testing.ReqnRoll)

Corvus.Testing.ReqnRoll can simplify the use of dependency injection. It can automatically create a dependency injection service collection (an instance of the IServiceCollection type defined by the Microsoft.Extensions.DependencyInjection.Abstractions component). It offers your tests an opportunity to populate this collection during the early phases of test execution. It then automatically builds an IServiceProvider from the service collection, and makes this accessible to your test. It disposes the provider at the end, ensuring any dependencies that need to clean up will have their Dispose methods called.

This feature offers two modes: you can either arrange for the service collection to be created, built, and torn down for individual scenarios, or you can create one that is created at the start of a feature, in which the service provider is shared by all scenarios within the feature, and the services are disposed only after all scenarios have run.

In most cases per-scenario mode is better, because it improves isolation between tests. However, if service initialization is particularly slow, feature-level containers might offer a useful escape hatch.

To use per-scenario mode, use the @perScenarioContainer tag. You can either put this on each of the individual scenarios you need, or put it at the top of the feature file, in which case every scenario in that feature will get its own container. To use per-feature mode, put a @perFeatureContainer tag at the top of the feature file. (You cannot use both modes at once.)

To populate the service collection you must write a binding that runs at the appropriate time. For per-scenario collections, it will look like this:

[BeforeScenario("@perScenarioContainer", Order = ContainerBeforeScenarioOrder.PopulateServiceCollection)]
public static void PopulateScenarioServiceCollection(ScenarioContext scenarioContext)
{
    ContainerBindings.ConfigureServices(scenarioContext, services =>
        services.AddLogging()
            .AddSingleton<IStore, FakeStore>());
}

and per-feature collections work in almost exactly the same way, largely just changing "Scenario" to "Feature":

[BeforeFeature("@perFeatureContainer", Order = ContainerBeforeFeatureOrder.PopulateServiceCollection)]
public static void PopulateFeatureServiceCollection(FeatureContext featureContext)
{
    ContainerBindings.ConfigureServices(featureContext, services =>
        services.AddLogging()
            .AddSingleton<IStore, FakeStore>());
}

The method name doesn't matter. It's the attribute that's significant. Note that by specifying the same @perScenarioContainer (or @perFeatureContainer) tag that Corvus.Testing.ReqnRoll is looking for, this binding will run for any tests that specify that tag. If you want to use this container feature in multiple tests but you want to use different service configuration in different tests, you can define your own tag, e.g. your feature file might start:

@perScenarioContainer
@adminTestContainerInitialization

And then you would specify that more specialized @adminTestContainerInitialization tag in your bindings instead of @perScenarioContainer.

To obtain services from the DI container, you can write code like this:

IServiceProvider serviceProvider = ContainerBindings.GetServiceProvider(this.scenarioContext);
MyService svc = this.ServiceProvider.GetRequiredService<MyService>()

The ContainerBindings.GetServiceProvider method is overloaded, accepting either a ScenarioContext or a FeatureContext. If you used the @perScenarioContainer tag you should pass the scenario context as the example above does, but if you're using @perFeatureContainer pass the feature context instead.

GetServiceProvider will work from inside any step, whether it's Given, When, or Then. But if you need to write a custom Before... binding that has access to the service provider, you'll need to make sure it runs at the appropriate moment. Since Corvus.Testing.ReqnRoll relies on this binding mechanism to create and initialize the services, you need to make sure that your code runs at the right moment, which means using the Order property on these ReqnRoll binding attributes. The bindings shown above that populate the IServiceCollection set their Order property with constants supplied by Corvus.Testing.ReqnRoll. This ensures that these bindings run after Corvus.Testing.ReqnRoll has created the service collection but before it has built the provider. If you want to access services from DI you will need to run after the provider has been built, which you can do by specifying a different constant for the Order:

[BeforeScenario("@adminTestContainerInitialization", Order = ContainerBeforeScenarioOrder.ServiceProviderAvailable)]
public static void ServiceProviderAvailableBeforeScenario(ScenarioContext scenarioContext)
{
    IServiceProvider serviceProvider = ContainerBindings.GetServiceProvider(scenarioContext);
    MyService svc = this.ServiceProvider.GetRequiredService<MyService>()
    // etc.
}

(This presumes that we've got some test-specific work going on. If you wanted this binding to run any time you used a per-scenario container you'd put @perFeatureContainer instead of @adminTestContainerInitialization.)

Note that the Order constants that Corvus.Testing.ReqnRoll provides space to allow you to control the order of multiple bindings of your own if necessary. The BuildServiceProvider is 9999 higher than PopulateServiceCollection, so if you need to perform multiple bindings in between the the service collection being created, and the final service provider being built from that collection, you can do so, and you can control their ordering. (E.g., you can have one binding with Order = ContainerBeforeScenarioOrder.PopulateServiceCollection, and then another with ContainerBeforeScenarioOrder.PopulateServiceCollection + 1.)

Exceptions during container disposal are handled using the Teardown Exception Handling mechanism described below, meaning that if errors occur, they will not prevent other teardown from happening, but will still eventually be reported. (Although be aware that depending on how you run your tests, feature-level errors will not necessarily be reported as errors, due to limitations inherent in how .NET test runners integrate with build and development tools.)

Child Object Value Retriever (Corvus.Testing.ReqnRoll)

If a scenario or feature specifies the @useChildObjects tag, this registers a Value Retrieved with ReqnRoll enabling named objects in the scenario context to be referred to with a {name} syntax. So if an earlier stage of a test puts something into the scenario context with the key transactionId you could write {transactionId} in a ReqnRoll feature fle to retrieve that value from the context and pass it in as the argument to a step.

Teardown Exception Handling (Corvus.Testing.ReqnRoll)

If you do any non-trivial work during After... bindings (e.g., tearing down a function) it is often important to ensure that all the bindings run. This is problematic in cases where failures might occur at this stage, because the only way a binding can report failure is by throwing an exception, and if it does so, it will normally prevent all other bindings from running, because it causes the test to come to an abrupt halt. (ReqnRoll doesn't have a concept of "fail, but continue to clean up".) This is especially problematic for integration tests that create external resources. (E.g., if your test creates resources in Azure, it can be costly if you fail to clean these up when you're done.)

Corvus.Testing.ReqnRoll provides a mechanism by which you can run code in some sort of After... binding, and be free to throw exceptions without that halting all further cleanup, while still eventually seeing the error. You do this through a helper like this:

[AfterScenario]
public void TeardownFunctionsAfterScenario()
{
    this.scenarioContext.RunAndStoreExceptions(this.functionsController.TeardownFunctions);
}

The RunAndStoreExceptions method here is an extension method on ScenarioContext. There's also one for FeatureContext.

This is the code in Corvus.Testing.AzureFunctions.ReqnRoll that tears down any functions created with the Azure Functions Launch feature described earlier. The code that performs this work is called TeardownFunctions and it's an instance member of the FunctionsController class, and it has been passed here as a delegate argument to RunAndStoreExceptions.

RunAndStoreExceptions lets you throw exceptions without risk. It catches any exceptions that emerge from your code, stores them, enables all other teardown to proceed, and then at the very last moment, it checks to see if any exceptions were detected this way, and if they were, it reports them all in one go by throwing an AggregateException containing every failure.

To work, Corvus.Testing.ReqnRoll defines unconditional AfterScenario and AfterFeature bindings each with an Order of int.MaxValue. This means they will only truly be the last thing to run for the scenario or feature if nothing else tries the same thing. If you want to use this feature, you will need to ensure that you're not using anything else that also depends on being able to be the very last thing that happens.

Licenses

Corvus.Testing is available under the Apache 2.0 open source license.

For any licensing questions, please email licensing@endjin.com

Project Sponsor

This project is sponsored by endjin, a UK based Microsoft Partner.

For more information about our products and services, or for commercial support of this project, please contact us.

We produce two free weekly newsletters; Azure Weekly for all things about the Microsoft Azure Platform, and Power BI Weekly.

Keep up with everything that's going on at endjin via our blog, follow us on X, or LinkedIn.

Our other Open Source projects can be found on GitHub

Code of conduct

This project has adopted a code of conduct adapted from the Contributor Covenant to clarify expected behavior in our community. This code of conduct has been adopted by many other projects. For more information see the Code of Conduct FAQ or contact hello@endjin.com with any additional questions or comments.

IP Maturity Scorecard

The IM is endjin's IP quality framework.