lundi 10 avril 2017

CasperJS test suite never fails & expected test count doesn't match but isn't recognized

When running a test suite, Casperjs used to correctly identify when a different number of tests/asserts were run than expected. (i.e. 4 asserts run, but casper.test.begin('...', 5, function...) should print out "expected 5 tests, but 4 ran") Now, no matter what number I put for the expected tests, it says "PASS ... (x tests) where x is the entered parameter for casper.test.begin.

I've stripped down to a minimal example and still the same thing happens. Below is my hello-test.js:

//hello-test.js
casper.test.begin("Hello, Test!", 7, function(test) {
    casper.start();
  casper.then(function() {
      test.assertEquals('a', 'c');

  });

  casper.then(function() {
      test.assertEquals('a', 'd');

  });

  casper.then(function() {
      test.assertEquals('a', 'e');

  });

  casper.then(function() {
      test.assertEquals('a', 'f');

  });
  casper.run(function() {
      test.done();
  });
});

Running casperjs test hello-test.js prints the following:

casperjs test hello-test.js 
Test file: hello-test.js                                                        
# Hello, Test!
FAIL Subject equals the expected value
#    type: assertEquals
#    file: hello-test.js
#    subject: "a"
#    expected: "c"
FAIL Subject equals the expected value
#    type: assertEquals
#    file: hello-test.js
#    subject: "a"
#    expected: "d"
FAIL Subject equals the expected value
#    type: assertEquals
#    file: hello-test.js
#    subject: "a"
#    expected: "e"
FAIL Subject equals the expected value
#    type: assertEquals
#    file: hello-test.js
#    subject: "a"
#    expected: "f"
PASS Hello, Test! (7 tests)
FAIL 4 tests executed in 0.11s, 0 passed, 4 failed, 0 dubious, 0 skipped.       

Details for the 4 failed tests:

In hello-test.js
  Hello, Test!
    assertEquals: Subject equals the expected value
In hello-test.js
  Hello, Test!
    assertEquals: Subject equals the expected value
In hello-test.js
  Hello, Test!
    assertEquals: Subject equals the expected value
In hello-test.js
  Hello, Test!
    assertEquals: Subject equals the expected value

Ideally, the suite should not have a PASS when one or more (in this example: every) test fails. It seems that Casperjs passes the suite if it reaches the end of the suite no matter the means. However, there should definitely be a mismatch between the 4 asserts and the expected 7, but Casperjs fails to print out the warning saying this.

What can I do to once again have the expected test count mismatch display again? Also, is there any way to have a test suite FAIL in the given casperjs version?

FYI: casperjs --version is 1.1.4 and phantomjs --version is 2.1.1

Aucun commentaire:

Enregistrer un commentaire