Run tests with Maven on pre-compiled project

Using Maven, I want to run unit and integration tests on a pre-compiled/ pre-jar'ed multi-module project. I am finding this exceedingly difficult.

My business case is, I want to compile and jar up a multi-module Maven project in one place, and then have the unit test and integration tests run using different Java versions and on different operating systems, to check for compatibility. Let's just say {Java 8, Java 10} x {Ubuntu, Windows} to keep it simple.

The reason I want to compile and jar up in one pace is, I want to ensure that I run tests on the actual code getting shipped. If I re-compiled everywhere I fear I might introduce errors unwittingly.

One solution is to do mvn deploy -DaltDeploymentRepository=$mydir on Ubuntu to produce a compiled project tree and a directory full of jars; then zipping up the project tree and deploy directory and shipping them off to a Windows machine; and there running mvn surefire:test -Dmaven.repo.local=$mydir on the project tree.

That works but it is hella clumsy. Surefire seems to read the test classes in the project tree and use the classes in the same project + jar dependencies from the deployment repo to link against. A problem there is, with -Dmaven.repo.local Maven needs to download all 3rd party dependencies again because that repo only contains my project artifacts. So a solid solution, but not great.

Instead what I have been trying to do is, just use the compiled project tree:

• If I just do mvn surefire:test I get problems resolving dependencies
• I have tried mvn compile -Dmaven.main.skip surefire:test to fix dependency resolution, which seems to work - just not for test dependencies
• I further tried mvn compile -Dmaven.main.skip test-compile -Dmaven.test.skip surefire:test, but for some reason that also does not work, the build again fails trying to resolve test dependencies.

Interestingly, it is not deterministic, the failures happen at different stages and sometimes not at all - I get the occasional green build!

And now I am at my wits' end: I keep thinking this should be simple, and that I can't possible be the first person struggling with this simple problem.

It is opaque to me what happens with the reactor, when and how it triggers. The dependency graph must have been well-formed, or it couldn't compile. So why can I not get Maven to reproduce the same dependency graph for running tests?

Regards,

Lasse

PS: This is related to what me esteemed colleague Ben has had problems with, too: How can I run integration tests after building all modules in a multi-module Maven project?