Why did my run fail to start?

You may have recently come across our updated Tabular Variable Warning message. The error message indicates that a run does not have enough available test data to complete the run. This results in the run immediately stopping.

What does the error message mean?

  • The warning is a proactive effort to alert you of when a run will fail.
  • The error message triggers when Rainforest detects tests that will fail due to an insufficient number of tabular variables to complete the tests.

Where does the message appear?

Rainforest will trigger:

  • An email that will include information about which variables are limited, along with the affected test numbers.
  • A red banner at the top of the browser.
  • A status error on the results page of the affected run.

Message examples:

What actions should I take?

Add more rows or modify your tabular variables.

Ensure you have enough variables for your test by calculating the number of necessary variables.

  • Number of test containing variable * number of browsers testing across * 6 (testers)
  • Ex: A run containing 3 tests that utilize an email login variable {{email.email_id}}, across Safari, Chrome, and Firefox (3 Browsers) would require 54 variables. 

Exclude affected tests from the run. 

  • By removing the test from the run, it will allow the run to start. If the affected tests are not addressed or removed, the run will not begin. 

FAQ:

Can I determine the specific tabular variable that the test ran out of?

  • Be on the look out for the email, it will list the limited tabular variables and an estimated number of variables needed.

Why do I need to account six testers?

  • Every test requires at least three testers to establish agreement if a test will pass or fail.
  • Additional testers are often recruited to ensure agreement, more details here.

Can I simply ignore and force the run?

  • No. The amount of tabular variables needed or the exclusions of those tests must be addressed.

Will a run that is currently in progress be aborted if I try to begin a second run that uses the same tabular variable(s) in the first run? 

  • No, tabular variables are allocated per run; thus a run that is currently in progress will not be affected by another that utilizes the same set of variables.
Did this answer your question?