FAQ

Can I determine the specific test data that the test ran out of?

Yes. The error message on the Run summary page and the email provide the names of the test data containing insufficient rows.

Why do I need to account for six testers?

Every test requires at least 2 testers to establish agreement on whether a test passes or fails. Additional testers might be recruited to ensure agreement.

Does a test run currently in progress fail if I start a second run that uses the same test data?

No. Test data is allocated on a per-run basis; a second run doesn’t affect the first.

What if I can’t add more test data, but I need to run my test cases right now?

Setting your test data as reusable allows you to run the test cases without adding more rows. Keep in mind that the number of concurrent testers running through your tests is limited to the number of rows available in any reusable test data. Reusable data has a “one out, one in” gate system, which can have a significant impact on your run time.

Does reusable test data affect multiple concurrent runs?

Yes. The reusable data still allocates the row to one tester at a time. The subsequent runs are queued, waiting for data to be freed up from the runs in progress.

Can I preform negative testing?

Use Test Data for Negative Testing

Yes. Negative testing helps to ensure your app can handle invalid data. For example, let’s say your sign-in flow calls for a password that is 6 characters long and contains at least 1 special character, such as an underscore or hash sign. If the password doesn’t meet the criteria, the app displays an error message.

To test whether the error triggers, you can instruct the test to key in {{random.number}}, which generates a 9-digit string in the password field. Then, observe to confirm that the error message is displayed.

If you have any questions, reach out to us at [email protected].