jeudi 18 juillet 2019

Testing different results with subprocesses

I'm using pytest tests. While running my test suite with

pytest /path_to_test/test.py

output (number of failed tests foremost) is robust, i.e. not changed.

But when I'm using -n auto flag in order to speed-up testing results differ between runs:

[01:04:24] $ pytest ~/Coding/Python/TEST/'tests (copy).py' -n auto -q --tb=no
Test session starts (platform: linux, Python 3.7.3, pytest 5.0.1, pytest-sugar 0.9.2)
rootdir: /home/beforeflight/Coding/Python/TEST
plugins: sugar-0.9.2, forked-1.0.2, xdist-1.29.0
bringing up nodes...

 tests (copy).py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                                                                    34% ███▌      

 tests (copy).py ⨯✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                                                                                                 48% ████▊     

 tests (copy).py ⨯✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                                              97% █████████▊

 tests (copy).py ⨯✓                                                                                                                                                                                   99% █████████▉

 tests (copy).py ⨯✓                                                                                                                                                                                  100% ██████████

Results (2.30s):
     140 passed
       4 failed
         - tests (copy).py:36 Tests.test_maybe_convert_objects[data9]
         - tests (copy).py:36 Tests.test_maybe_convert_objects[data4]
         - tests (copy).py:36 Tests.test_maybe_convert_objects[data125]
         - tests (copy).py:36 Tests.test_maybe_convert_objects[data127]
[01:04:27] $ pytest ~/Coding/Python/TEST/'tests (copy).py' -n auto -q --tb=no
Test session starts (platform: linux, Python 3.7.3, pytest 5.0.1, pytest-sugar 0.9.2)
rootdir: /home/beforeflight/Coding/Python/TEST
plugins: sugar-0.9.2, forked-1.0.2, xdist-1.29.0
bringing up nodes...

 tests (copy).py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                                                            40% ███▉      

 tests (copy).py ⨯✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                                                                                                  53% █████▍    

 tests (copy).py ⨯✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                                                100% ██████████

Results (2.27s):
     142 passed
       2 failed
         - tests (copy).py:36 Tests.test_maybe_convert_objects[data9]
         - tests (copy).py:36 Tests.test_maybe_convert_objects[data4]

So here number of failed tests change between 2-4 (but in 2-6 actually). Why is that?

Aucun commentaire:

Enregistrer un commentaire