Best UI test that run without opening brawser, for java

I have general question about testing. I want to create test project for UI web testing(automates browsers testing), the requirement is:

1- The test project should not open any browser.

2- The test project should test 3 browsers(Chrome, Firefox, safari).

3- The test project most be in java language.

I have tried selenium, uncertainty it dose not work without opening browser. I tried selenium headless it also dose not work well.

Which developing environment is good for this requirements?