How to Debug Automation Results

Learn how to troubleshoot your tests using Automation results.

The Results page provides valuable information on tests at the step level. However, if you want to gain a better understanding of how each step ran, you can use the Investigate Action feature. To view details on any step, click Investigate Action.

Any text value used during execution, such as URL, email, and username, can also be copied from the results by highlighting the text in the test step and copying it.

To return to the test results, click the Back arrow at the top of the page.

11971197

Investigating actions.

Reviewing a Passed Step

Sometimes, you might find it helpful to see how a match occurred in a step that passed. You can see the following information:

  • Attempted to find a match (image)
  • Closest match identified (image)
  • Results determined
  • Metadata
  • The Closest match location is shown in the screenshot to the right.
13451345

Detailed step results.

If a visual match was performed using pixel data, detailed information on how close the match was is shown in the “Result determined” section.

350350

An element matched using visual match.

If a match was performed using Text Matching, the text is shown in the “Result determined” section. See Text Matching for more information.

349349

An element matched using Text Matching.

Reviewing a Failed Step

When you click the Investigate Action link on a failed step, you see the same information that displays for a passed test, except that neither visual nor text matching was successful.

Information, including the closest visual match and the text used to match (if applicable), is displayed.

Test Retries

Sometimes, tests fail due to reasons that include network, load, and intermittent bug issues. The Test Retries feature allows you to automatically rerun failed automation tests up to 3 additional times to pass the test. For more information on this feature, see Test Retries

Image Comparison and Update

For failed steps, you can investigate further by comparing the requested and closest matched items.

  1. Click the Can’t see the difference? link under “Closest match identified” to compare the images.
  2. Use Swipe or Layered to determine whether the closest match can be used.
  3. When you’re finished, click Done.
832832

Comparing image differences.

  1. To update the failed step directly from the result, confirm the suggested match is appropriate to use, then click the Accept Replacement button.
    Note: the test is automatically updated.
515515

Select Update screenshot to use the suggested image.

If the closest match isn’t acceptable, then open the test, edit the step, and capture the correct element.

The Site Can’t Be Reached When Navigating to the Test URL

You might find you cannot get to the site. Make sure to whitelist all Rainforest IP addresses. See Which IP Addresses Do Rainforest Tests Come From? for more information.

15801580

Example of a blocked IP address.

Review Video

Video information detailing execution is available for review. To play:

  1. Select Play icon in the middle of the VM
  2. Use controls at bottom to control the playback including the speed.

Each green dot on the playback represents an action execution. Hover over it to see which action it is and click to advance to that action.

To make triaging easier, Click actions are highlighted using a ripple effect to indicate the target.

11671167

Results video with click highlights.

Troubleshooting with HTTP, Browser, and Video Logs

With HTTP logs, you can view all HTTP traffic passing to and from each Rainforest virtual machine (VM). Use the logs to examine request interruptions at the network level and see the order in which HTTP requests were made.

To view a test’s HTTP or browser logs, or a recording of the video results:

  1. Click the test’s run results.
  2. Click the Export dropdown.
  3. Select “HTTP Logs,” “Browser Logs,” or “Video” to download the files.

View the HAR and browser files in your text editor of choice, using this recommended tool or Google Chrome extension.

26962696

Getting HTTP logs for troubleshooting.


If you have any questions, reach out to us at [email protected].


Did this page help you?