Android instrumented test not starting an app on physical device

I'm trying to write simple UI tests using Espresso. I have test class in androidTest:

@RunWith(AndroidJUnit4.class)
public class AFragmentTest {

    @Rule
    public ActivityScenarioRule<MainActivity> activityScenarioRule = new ActivityScenarioRule<MainActivity>(MainActivity.class);

    @Test
    public void test() {
        activityScenarioRule.getScenario().moveToState(Lifecycle.State.RESUMED);
        FragmentScenario<AFragment> fragmentScenario = FragmentScenario.launchInContainer(AFragment.class, null, R.style.AppTheme, null);
        fragmentScenario.moveToState(Lifecycle.State.RESUMED);

        //some espresso code
    }
}

When I'm running this on emulator, the app opens, tests are executed and the app closes. On physical device however, when I run the test, it stuck on "Running test..." until I manually run the app. If there are multiple tests, I have to run app manually every single test.
Do you have any idea how to fix this?
Thank you.