Intellij always asks for Gradle task selection for test run

I have two different test tasks in my Gradle project, test for unit tests and integrationTest for integration tests. All integration tests are annotated with @Tag("integration") and the test and integrationTest tasks respectively ex- or include tagged tests. This works fine as long as I execute the gradle tasks directly but whenever I execute the tests directly in Intellij via the green icon left of each test I get constantly asked which task I want to execute.

Is there a way to make Intellij recognize the correct task for each test based on its tag? Or at least make it cache my decision?