lundi 16 novembre 2020

Count each subtest for "Failed %" in MSTest's trx file

We are running an automated test where each file counts as its own test. We are using the Dynamic Data attribute to provide files that need to be tested. Currently, each file gets tested, however in the TRX file, it is logging them, essentially, as sub-tests, and if any 1 test fails, then the whole bucket counts as failing. This gives us inaccurate reads as far as how many files actually failed or passed (because if one failed, then the entire thing gets marked as failing) when we publish our test results in the azure pipeline. Is there a way to mark these subtests as actual tests so that the counting is done accurately?

Here you can see 2 of the subtests actually passed, however that is not reflected in the results in the header (note that the reason it say 0/4 instead of 0/1 is that there are 3 other similar test "buckets" that have passing and failing tests but are also being marked as all failing.

Aucun commentaire:

Enregistrer un commentaire