pytest-randomly - detect test coupling

Summarize the problem:

I have ~1000 tests, tested with pytest-randomly. I know they fail due to coupling, when --randomly-seed=42, for Example:

test_1 OK
test_2 OK
...
test_x OK [the one I want to find]
...
test_y FAILED [ y and x are coupled ]
...
test_1000

I'd like to automatically find the smallest test sequence, that creates the coupling (for this example, it would be test_x -> test_y). Is there a way to do it?

What you've tried

• Running the whole test suite until I found, that seed 42 creates an error.

• I think this is solvable by running test cases 1000 times in --randomly-seed=42 order - each time excluding one test, and checking if tests still fail. If they fail, the excluded test is a noise, if they don't, the excluded test is necessary to reproduce the error. This way, after 1000 runs I should have a much smaller set of tests causing the error. But this sounds like something clearly already implemented.

• Other problem would be - how to recreate the exact test ordering, but without one chosen test?

• I am basically looking for something that would look through a given sequence... similar way git rerere goes through git commits.