How to Debug Automation Results

Overview

The run page makes it easy to quickly review and take action on failures. You can see key details for quick failure assessment while still allowing you to dig deeper when needed with things like comparison match and provided logs. The page also shows self-healed blocks directly on the run page allowing you to make a more informed decision about whether it’s necessary to jump to the video and full step list in the editor.

Reviewing a Failed Step

From the run page, click on a failed test's row to view details on why the test failed. Expanding the failure shows a detailed view with the following information:

• Screen during failure: the screen searched during the failing step
• Compare match: compare the target element to the closest match found
  • To accept the closest match found, click Set as target... and crop and review to save the changes to the step.

To take a closer look, click Open to view the test result in more detail or make additional changes outside of the failed step.

Test Retries

Sometimes, tests fail due to reasons including network, load, and intermittent bug issues. The Test Retries feature allows you to automatically rerun failed automation tests up to 3 additional times to attempt to pass the test. For more information on this feature, see Test Retries

Image Comparison and Update

For failed steps, you can investigate further by comparing the target and closest-matched images.

1. Click Compare match on the expanded test failure.
2. Use the Swipe or Layered options to determine whether the closest match can be used.
3. When you determine what changes must be made, make the appropriate changes to the test.

The Site Can’t Be Reached When Navigating to the Test URL

If you find the Rainforest virtual machine can't reach your website, make sure to whitelist all Rainforest IP addresses. See Which IP Addresses Do Rainforest Tests Come From? for more information.

Troubleshooting with HTTP, Browser, and Video Logs

With HTTP logs, you can view all HTTP traffic that passed to and from each Rainforest virtual machine (VM) during a test. Use the logs to examine request interruptions at the network level and see the order in which HTTP requests were made.

To view a test’s HTTP logs, browser logs, or a recording of the video results:

1. Click the Open button on the test's row on the run page
2. Click the 3-dot icon at the top right of the page.
3. Select “HTTP Logs,” “Browser Logs,” or “Video” to download the relevant file.

View the HAR and browser files in your text editor of choice, using this recommended tool or Google Chrome extension.

If you have any questions, reach out to us at [email protected].