Intellij insists on using Android JUnit instead of Gradle for tests

I have a multi-module gradle project that includes some core backend modules, as well as different application modules for different platforms. For example, one android app module, and one module that just runs as a CLI.

I have configured gradle to run tests by going to Build, Execution, Deployment -> Gradle -> Runner and both checking the Delegate IDE build/run actions to gradle checkbox and selecting Gradle Test Runner from the Run tests using dropdown.

I have refreshed Gradle multiple times, as well as invalidating caches / restart. Despite this, when I click the run icon in the gutter for tests, even in the non-android modules, Intellij creates an Android JUnit configuration to run the test. This ends up not rebuilding after changes are made.

This did not used to happen, though I'm not sure when it started behaving this way or what changed recently.

How can I force IJ to always use gradle? I'd rather have no Android JUnit available, even in the android module, than the current behavior.