Why did my run fail to start?

  • The run does not have enough available test data to complete the run. 

What does the error message mean?

  • The error message triggers when Rainforest detects tests that will fail due to an insufficient number of tabular variables to complete the tests

Where does the message appear?

Rainforest will trigger:

  • An email that will include information on which variables are limited, along with the affected test numbers.
  • A red banner at the top of the browser within Rainforest.
  • A status error on the results page of the affected run.

Message example:

What actions should I take?

Add more rows or modify your tabular variables

Ensure that you have enough variables for your test by calculating the number of necessary variables.

  • Number of tests containing variable X number of browsers testing across X 6 (testers)
  • Ex: A run containing 3 tests that utilize an email login variable {{email.email_id}}, across Safari, Chrome, and Firefox (3 Browsers) would require 54 variable values.

    3 Tests X 3 Browsers X 6 Testers = 54 variable values

Exclude affected tests from the run

  • By removing the test from the run, it will allow the run to start. If the affected tests are not addressed or removed, the run will not begin.

FAQ

Can I determine the specific tabular variable that the test ran out of?

  • Be on the look out for the email, it will list the limited tabular variables and an estimated number of variables needed

Why do I need to account for six testers?

  • Every test requires at least three testers to establish agreement if a test will pass or fail.
  • Additional testers may be recruited to ensure agreement, more details here.

Can I simply ignore and force the run?

  • No, you must be address the tabular variable warning. 

Will a run that is currently in progress be aborted if I try to begin a second run that uses the same tabular variable(s) in the first run? 

  • No, tabular variables are allocated per run; thus a run that is currently in progress will not be affected by another that utilizes the same set of variables.

What if I can't add more variables, but I need to run my test cases right now?

  • Setting your variable as reusable will allow you to run the test cases without adding more rows, but keep in mind that the number of concurrent testers running through your tests will be limited to the number of rows available in any reusable tabular variable. Reusable variables have a "one out, one in" gate system, which can have a significant impact on your run time.
Did this answer your question?