samedi 10 avril 2021

How to flag unexpected TODO tests passing using yath?

When a test flagged as 'todo' starts passing, under the 'prove' harness, the unexpected pass is flagged. The yath harness doesn't flag the unexpected pass. Is there some way to convince yath to behave like prove?

$ cat todo.t 
#!/usr/bin/env perl
use Test2::V0;

{
    my $todo = todo "Todo tests";

    is(1, 2, "FAILING test");
    is(1, 1, "PASSING test");
}

done_testing;
$ prove todo.t 
todo.t .. ok   
All tests successful.

Test Summary Report
-------------------
todo.t (Wstat: 0 Tests: 2 Failed: 0)
  TODO passed:   2
Files=1, Tests=2,  0 wallclock secs ( 0.01 usr  0.00 sys +  0.07 cusr  0.00 csys =  0.08 CPU)
Result: PASS
$ yath todo.t 

** Defaulting to the 'test' command **

(  DIAG  )  job  1    +-----+----+-------+
(  DIAG  )  job  1    | GOT | OP | CHECK |
(  DIAG  )  job  1    +-----+----+-------+
(  DIAG  )  job  1    | 1   | eq | 2     |
(  DIAG  )  job  1    +-----+----+-------+
( PASSED )  job  1    todo.t

                                Yath Result Summary
-----------------------------------------------------------------------------------
     File Count: 1
Assertion Count: 2
      Wall Time: 0.36 seconds
       CPU Time: 0.57 seconds (usr: 0.18s | sys: 0.02s | cusr: 0.32s | csys: 0.05s)
      CPU Usage: 156%
    -->  Result: PASSED  <--

Verbose output is a little more helpful, but it's, y'know, verbose. If I screw my eyes up I can see the unexpected passing test didn't generate any error diagnostics and that's the only indication.

$ yath -v todo.t 

** Defaulting to the 'test' command **

( LAUNCH )  job  1    todo.t
(  NOTE  )  job  1    Seeded srand with seed '20210410' from local date.
[! PASS !]  job  1  + FAILING test
[  DEBUG ]  job  1    todo.t line 7
{  TODO  }  job  1    Todo tests
(  DIAG  )  job  1    +-----+----+-------+
(  DIAG  )  job  1    | GOT | OP | CHECK |
(  DIAG  )  job  1    +-----+----+-------+
(  DIAG  )  job  1    | 1   | eq | 2     |
(  DIAG  )  job  1    +-----+----+-------+
[! PASS !]  job  1  + PASSING test
{  TODO  }  job  1    Todo tests
[  PLAN  ]  job  1    Expected assertions: 2
( MEMORY )  job  1    rss:  24852kB
( MEMORY )  job  1    size: 36640kB
( MEMORY )  job  1    peak: 36640kB
( PASSED )  job  1    todo.t
(  TIME  )  job  1    Startup: 0.07232s | Events: 0.00119s | Cleanup: 0.00286s | Total: 0.07637s

                                Yath Result Summary
-----------------------------------------------------------------------------------
     File Count: 1
Assertion Count: 2
      Wall Time: 0.34 seconds
       CPU Time: 0.55 seconds (usr: 0.21s | sys: 0.01s | cusr: 0.29s | csys: 0.04s)
      CPU Usage: 159%
    -->  Result: PASSED  <--

Aucun commentaire:

Enregistrer un commentaire