Why did my run error?

  • The error message triggers when a tabular variable has an insufficient number of rows to assign to all testers needed for the run.

Where does the message appear?

Rainforest will trigger:

  • An email that will alert you to the run error due to the variable, including a link to the run and the name of the tabular variable with insufficient numbers

    Note: you will need to enable email notifications to receive run error notifications in your account settings.
  • A status error on the results page of the affected run including the name of the tabular variable with insufficient numbers and a link to the step variable settings

Email example:

Run summary error example:

What actions should I take?

Add more rows or modify your tabular variables

Ensure that you have enough variables for your test by calculating the number of necessary variables.

  • Number of tests containing variable * number of browsers being tested * 6 (testers)
  • Ex: A run containing 3 tests that utilize an email login variable {{email.email_id}}, across Safari, Chrome, and Firefox (3 Browsers) would require 54 variable values.

    3 Tests * 3 Browsers * 6 Testers = 54 variable values

Exclude affected tests from the run

  • If the affected tests are not addressed or removed, the run may not complete.

FAQ

Can I determine the specific tabular variable that the test ran out of?

  • Both the error message on the Run summary page and the email will provide the tabular variable names that have insufficient numbers available

Why do I need to account for six testers?

  • Every test requires at least three testers to establish agreement if a test will pass or fail.
  • Additional testers may be recruited to ensure agreement, more details here.

Will a run that is currently in progress result in an error if I try to begin a second run that uses the same tabular variable(s) as the first run?

  • No, tabular variables are allocated per run; thus a run that is currently in progress will not be affected by another that utilizes the same set of variables.

What if I can't add more variables, but I need to run my test cases right now?

  • Setting your variable as reusable will allow you to run the test cases without adding more rows, but keep in mind that the number of concurrent testers running through your tests will be limited to the number of rows available in any reusable tabular variable. Reusable variables have a "one out, one in" gate system, which can have a significant impact on your run time.
Did this answer your question?