jeudi 17 août 2017

Running Fixtures but not methods in parallel NUnit

I have got a question regarding test parallelization with parameterized fixtures and methods using NUnit 3.8.0.0, .NET 4.0 and C#.

I am currently developing a test suite that runs tests using each resource in a set of resources. Since initializing these resources is quite time consuming, and the resources do not preserve state, I want to share them between my test cases, in order to improve performance. These resource can not be used simultaneously by multiple tests.

In order to do that, I made a base fixture with a constuctor taking one argument, which makes the fixture point to the right resource. The actual test fixtures subclass from this fixture.

I am having a test fixture that uses both constuctor arguments (passed to a base class) and test case arguments. Based on the parameter, the base class initializes some resources. These resources can't be used at the same time.

Now I am trying to parallelize these test cases in such a way that the tests in the different fixtures generated by the class MyParameterizedTestFixture(V1), MyParameterizedTestFixture(V2). can run simultaniously, but tests in the same fixture can not (since the resources can not be used simultaniously). Additionally, different fixture classes may not run in parallel.

My code looks like this (with details abstracted away):

[TestFixture]
[TestFixtureSource(typeof(TestData), "FixtureParams")]
public class MyParameterizedTestFixture : BaseFixture
{
     public MyParameterizedTestFixture(Resource resource) : base (resource) { }

     [Test]
     public void Test1() { /* run test using Resource */ }

     [TestCaseSource("Params")]
     public void TestParameterized(object param) { /* run test using Resource and param */ }
}

I tried to add Parallelizable(ParallelScope.Self)] to my derived fixtures, but that results in tests from different fixtures using the same resource (i.e. having the same version) fixture parameter, being run simultaniously (it works when only selecting one fixture).

Using [Parallelizable(ParallelScope.Fixture)] is definately not correct, because it will cause different fixtures to run together. Using [Parallelizable(ParallelScope.Children)] is also not correct, because it will cause different test cases in a fixture to run together, and not the different fixtures from the same class.

Now I was wondering if something like what I want could be archived by using a 'layered' approach (marking fixtures parallelizable some way, and methods in another way), or is it possible to define custom parallel scopes?

Aucun commentaire:

Enregistrer un commentaire