Block inside pytest.raises context manager not getting executed

I have been writing a tokeniser for a little language I'm developing in my free time, and I want to test it with negative input. My code can raise two kinds of Exceptions, an IndentationError and a SyntaxError and it kind of looks like this.

class Tokenizer:
    def __init__(self, source:str) -> None:
        self.source = source
        ...

    def get_next_token(self) -> Token:
        """Returns the next token and advances the position pointer into self.source"""
        ...

    def tokenize(self) -> Generator[Token, None, None]:
        """Calls get_next_token until all of self.source has been scanned"
        ...

Then in the tests I have lists with all sorts of input. For example:

SYNTAX_ERROR_SOURCES = ["(}", "[}", "{]", "a@b"] # Note that @ is not a supported operand

And then one of the test functions looks like this:

@pytest.mark.parametrize("source", SYNTAX_ERROR_SOURCES)
def test_tokenize_fail_syntax_error(source: str) -> None:
    with pytest.raises(SyntaxError):
        Tokenizer(source).tokenize()

The problem is that tokenize never gets called inside the context manager and I'm not sure why. I can confirm that, by running the code under a debugger. When it reaches the line, where tokenize gets called, it directly goes to some pytest code and after that the test fails with Failed: DID NOT RAISE <class 'SyntaxError'> without ever going into tokenize.

Also, if I execute tokenizer.py as a script with python tokenizer.py "a@b" then everything is fine and the program exits with a SyntaxError, as expected.

Has anyone ever come across something like this?