Best Practices

Running Tests on One Web Browser

Run automated tests with the browser they were created on. Since our Automation Service relies on screenshot matching, the tests are optimized for a single browser. Note that the Tester Community can run these tests using multiple browsers.

Grouping Tests by Browser

Group each browser’s tests into a separate Run Group. This ensures the tests only run using the browser they were created for.

Capturing Quality Screenshots

Capture Small, Precise Screenshots

Capturing a more focused, smaller screenshot reduces test brittleness and maintenance. A more focused screenshot ensures that the element can still be matched if there are changes to the surrounding areas. The automation service scans for a 95% pixel match to pass the step with a visual match. This allows for some flexibility for minor pixel differences, but small differences may not be detected so precise screenshots are key.

Capture Unique Screenshots

Reduce “flakiness” by capturing unique UI elements. If an element or text appears multiple times on one page, capture a unique screenshot by including surrounding areas to differentiate the elements.

Capture Hover Elements

For elements you need to hover over, capture them while they’re in a nonactive state to ensure the hover behavior is recorded.

To capture a screenshot of an element that only appears during a hover, use the Preview Actions button. Doing so makes the automation agent hover for you to capture the screenshot of the following action.

Assign Effective Element Names

If you choose to run your tests with the Tester Community, the testers rely on element names as descriptors for the test steps.

Add Wait and Observe Actions

The automation agents might execute an action faster than a page can load. Reduce flakiness by adding strategic slow-downs with the Wait action or Observe action. Doing so ensures the element appears before the test can continue.

If you have any questions, reach out to us at [email protected].