mardi 7 mai 2019

Locking shared resources in parallelized End-to-End Tests

We are using NUnit to conduct End-To-End tests on our MVC API services. Many of these tests write and read/assert on data in shared systems. In order to avoid state corruption, tests are executed sequentially. This is painfully slow.

Theoretically this can be parallized by assigning a separate User/Principal to each test. Data is (mostly) isolated per User... However Users are on limited supply. We've got about 10 Users available. Meaning 10 Tests could be running in parallel at any one time.

Now I don't see NUnit offering any kind of "Semaphore Lock" to facilitate above testing process. And I wonder how it can and should be achieved? Does a feature like this exist already?

Ideally it would look something like this:

[TestFixture]
[Parallelizable(ParallelScope.All)]
public class TestClass
{
    [Test]
    [LockPoolObject(typeof(UserPool)]
    public void Test_Method_1(UserPool.User testUser)
    {
        // Dispatch request using testUser
    }

}


Where [LockPoolObject] tells NUnit to instantiate passed in type only ever once and call whatever method to retrieve/wait on one of the pooled objects plus disposes of it when the test completes. UserPool makes the shared Users available on instantiation.

Thoughts: Effectively a User is some kind of ParallelScope and the desired behaviour might be achieved by reordering all test cases into fixtures per User. This would completely break our logical test structure however...

Aucun commentaire:

Enregistrer un commentaire