Jest complains Cannot find module when using Yarn Link

So I have a components library that a repo uses to import its UI components. When I yarn link the library, every change works with the repo except for unit-testing with jest

For some reason jest is not "smart enough" to detect package being yarn linked and therefore the tests fails.

The output of the errors is Cannot find module '....' from '...'

A workaround is doing a pre-release publish of the components library but that is NOT the solution and doesn't tackle the problem itself. I work at a company so "hacky" solutions will not be accepted.

Did anyone came across this problem and have a way to solve this?