jeudi 6 juillet 2017

Instrumentation Tests: define the first test as mandatory for the following

I have an Android instrumentation test which tests my ServerApi and multiple other instrumentations which test e.g. my Activities.

One of the reasons the ServerApi test can fail is (sadly) a broken wifi. In this case, my whole test suite takes ages to complete because all the activities dealing with network access have to run into timeouts before failing.

How can I define the ServerApiTest to run in the beginning and if it fails, cancelling all other following tests?

public class ServerApiTest extends ApplicationTestCase<Application> {
    //...

    public ServerApiTest() {
        super(Application.class);
    }

    @Override
    public void setUp() throws Exception {
        Log.i(TAG, SERVER_KEYWORD + ": prepare");
        super.setUp();
        _context = getContext();
    }

    @Override
    public void tearDown() throws Exception {
        Log.i(TAG, SERVER_KEYWORD + ": cleanUp");
        super.tearDown();
    }

    public void testPing() throws InterruptedException, ExecutionException,
                                  URISyntaxException {
        //...
    }

As an example an excerpt of one of the Activity tests:

@RunWith(AndroidJUnit4.class)
@MediumTest
public class RecordingActivityTest {
    private static final String TAG = RecordingActivityTest.class.getSimpleName();

    @Rule
    public ActivityTestRule<RecordingActivity> _activityRule
        = new ActivityTestRule<>(RecordingActivity.class);


    @Test
    public void testOnCreate() {
        onView(withId(R.id.lytRecordingButtons)).check(
               matches(withEffectiveVisibility(ViewMatchers.Visibility.VISIBLE)));
    }
    //...

Interestingly, I have observed in the test reports that on some (rare and yet to be explained) occasions if some test fails, the test suite indeed seems to abort.

See the screenshots: the RecordingActivityTest failed and the number of tests is reported as 25. But with no changes in the tests (only fixing the bug which caused RecordingActivityTest to fail), the next build correctly showed all 43 tests again - coincidentally now with the tests failing due to a broken WiFi.

First test run which does not count all tests: First test run which does not count all tests Following test run which shows all tests again: Following test run which shows all tests again

Whatever caused the tests not to be executed, could I use this mechanism for what I want to achieve?

Aucun commentaire:

Enregistrer un commentaire