mercredi 20 mai 2020

how can assert exception message with pytest xfail (assertion on exception type is working)

Let's assume I have a file test_scratch_2.py


import pytest


def my_fun(number: int):
    if number == 1:
        raise ValueError("Some Message for number 1")

    if number == 2:
        raise ValueError("Some Message for number 2")

    if number == 3:
        return int("number")

    return number ** 2


@pytest.mark.parametrize("digit", [
    # example 1
    pytest.param(1, id="first",
                 marks=pytest.mark.xfail(raises=ValueError,
                                         strict=True,
                                         reason="Some Message for number 1")),
    # example 2
    pytest.param(2, id="second",
                 marks=pytest.mark.xfail(raises=ValueError,
                                         strict=True,
                                         reason="Some Message for number 1")),
    pytest.param(3, id="third",
                 marks=pytest.mark.xfail(raises=ValueError,
                                         strict=True,
                                         reason="Some Message for number xxxxxxxxx")),

    4,
    5,
    60000
])
def test_my_fun(digit):
    assert my_fun(digit) > 0

And I run the tests

> pytest test_scratch_2.py -vvv                                                                                                                                                                                                
======================= test session starts ============================
platform linux -- Python 3.7.5, pytest-5.3.5, py-1.8.1, pluggy-0.13.1 --.../bin/python3
cachedir: .pytest_cache
rootdir:....config/JetBrains/PyCharm2020.1/scratches
collected 6 items                                                                                                                                                                                                             

test_scratch_2.py::test_my_fun[first] XFAIL                                                                                                                                                                             [ 16%]
test_scratch_2.py::test_my_fun[second] XFAIL                                                                                                                                                                            [ 33%]
test_scratch_2.py::test_my_fun[third] XFAIL                                                                                                                                                                             [ 50%]
test_scratch_2.py::test_my_fun[4] PASSED                                                                                                                                                                                [ 66%]
test_scratch_2.py::test_my_fun[5] PASSED                                                                                                                                                                              [ 83%]
test_scratch_2.py::test_my_fun[60000] PASSED   

There is a bug in case of test_scratch_2.py::test_my_fun[third] XFAIL.

It fails with an expected exception but not with the expected reason.

All assertions are based on exception type and not on exception messages.

How I could check the exception message?

Aucun commentaire:

Enregistrer un commentaire