testing suite for lots of test cases

i am looking for a workflow test suite. Imagine i do have many services processing events, as a whole system from a very toplevel view it looks like this:

{ some input data } ---> processing via multiple services ---> { result }

So if i want to test my input to my expected output, for example my test set does have more than 100k test cases associated with certain results. What would be a good test suite (open source), i can use to handle this, i would like to do this test as part of my ci/cd in jenkins. I guess pytest is not the best way here, for doing some sort of integration / data testing. What do you recommend in this scenario? Maybe a tool from the spectrum of machine learing test suites?

I do code with python currently on the side node.

Thanks in advance