Why does a single module import add 1.5 seconds to a pytest run?

I am writing some code with TDD and have a test that looks like this (note I am using pytest-describe, but the same issue exists when using my tests without the additional formatting):

import numpy as np
import pytest
from maszcal.emulator import LensingEmulator, LargeErrorWarning




class FakeInterpolator:
    def __init__(self, coords, grid):
        pass

    def process(self):
        pass

    def interp(self, coords):
        return np.ones(tuple(c.size for c in coords))


def describe_emulator():

    def describe_error_check():

        @pytest.fixture
        def emulator(mocker):
            mocker.patch('maszcal.emulator.RbfInterpolator', new=FakeInterpolator)
            lensing_emulator = LensingEmulator()
            lensing_emulator.generate_grid = lambda coords: np.ones(tuple(c.size for c in coords))
            return lensing_emulator

        def other_test_1(emulator):
            do_stuff_with_emulator...

This test takes 2.5-3 seconds to run even with pytest --collectonly. When LensingEmulator gets imported, it imports from the maszcal.interpolate module like so:

spec/emulator/LensingEmulator.py
    imports ↑  
maszcal/emulator.py
    imports ↑
maszcal/interpolate.py
    imports ↑
          GPy

where GPy is an external module. Of course there are other dependencies, but what is odd is that if I delete the import GPy line from maszcal.interpolate I get a speedup to 0.9ish seconds instead of 2.5-3.

I am fairly confused by this behavior because importing the module by itself is not that slow, but it adds a ridiculous amount of time the collection phase of the test. What is the proper way to address this issue?