I had written some tests using for loops and recently discovered this handy doc page that describes how to write data-driven tests.
https://devexpress.github.io/testcafe/documentation/recipes/create-data-driven-tests.html
I'm now trying to refactor my tests but running into a problem. The purpose of the test is to log in as a bunch of different accounts, and then verify whether certain page elements exist. (I recognize that this is a heavy hammer to use, but our app has a huge number of permissions and the combinations often get forgotten when developing new features, so this seemed like the quickest way to get at the actual truth of what is on the screen for a real user).
My old tests look something like this:
test('Account manager', async (t) => {
const existingItems = [
[mediaSidePanel.customize, 'Customize'],
[mediaSidePanel.stats, 'Stats'],
[mediaSidePanel.download, 'Download'],
[mediaSidePanel.delete, 'Delete'],
];
const notExistingItems = [
[mediaSidePanel.adminTools, 'Admin Tools'],
];
await t
.useRole(advAccountManager)
.navigateTo(`https://${accountKey}.wistia.io/medias/${mediaHashedID}`);
await Promise.all(existingItems.map(async item => await t
.expect(item[0].exists).ok(`${item[1]} should exist for an Account Manager`)));
await Promise.all(notExistingItems.map(async item => await t
.expect(item[0].exists).notOk(`${item[1]} should not exist for an Account Manager`)));
});
The test works fine except for the obvious problems of having loops in tests: I need to have thorough diagnostic messages to ensure I know which element actually is failing, and worse, if something early in the loop fails, the test ends, and I have no way of knowing if there would have been subsequent failures.
I started trying to refactor this by pulling all of the existing/non-existing items into an array defined in a separate file and wrote this:
import * as dataSet from '../helpers/rolePermissions';
fixture `Advanced Account Manager`
.page `https://app.wistia.io/logout`
.beforeEach(async (t) => {
await t
.useRole(advAccountManager);
});
dataSet.advAccountManager.project.forEach(data => {
test.page `https://${accountKey}.wistia.io/projects/${projectHashedID}`(`Project - ${data.name}`, async t => {
if (data.present) {
await t
.expect(await data.selector.exists).ok(`${data.name} should exist for an Account Manager`);
}
else {
await t
.expect(await data.selector.exists).notOk(`${data.name} should not exist for an Account Manager`);
}
});
});
It works perfectly in that it gets rid of the biggest problem and keeps running the tests even if an earlier one fails. It introduces a much bigger problem, however. This is now considerably slower because it has to log in every single time it iterates through the test. I am already using Roles, as you can see, to try to speed things up, but it is still painfully slow comparatively. I don't want to continue down this refactoring path if it ultimately isn't going to pan out.
Is there a way to have the best of both worlds? Ideally, I would like to do the following:
- log in once at the beginning of the fixture
- stay on the page without reloading
- iterate through all the relevant selectors
- continue iterating even if an individual test fails
