How to Match UI Elements Using Multiple Methods

Overview

Web application user interfaces often change, even when the underlying functionality doesn't. These changes can break tests and require time-consuming test maintenance in some test automation tools.

Rainforest has multiple fallback methods for identifying target elements, making your tests more robust and less likely to fail when you make non-functional changes to your app.

How it works

When you create steps in your test, we’ll prompt you to use the following matching methods:

  • Screenshot: look for the target element visually
  • DOM selector: look for the underlying code corresponding to the target element
  • AI: look for the target element based on its AI-generated description

Rainforest will match a UI element using the methods you select.

Using Rainforest-generated selectors and AI labels

If you’re writing your test based on Chrome on Windows, when you create a screenshot of an element, Rainforest will attempt to generate (1) a DOM selector and (2) an AI description of the element. By default, all methods are selected.

How matching methods are evaluated

When a step is run, Rainforest will try to find an element that matches the first selected method (e.g., a screenshot). If Rainforest fails to find the element using this method, it'll try to find the element using the next enabled method. This repeats until Rainforest either finds the element by one of the enabled methods (i.e., the step passes) or fails to find the element across all enabled methods (i.e., the step fails).

How to edit the matching method

Hover over an element's screenshot in a test step to see the popover with the matching settings. Here, you can edit details for your screenshot (including adding a mask), edit the DOM selector, and change the AI description.

Reviewing results with multiple matching methods

When reviewing details for a failed step, you’ll be able to see which methods were used when Rainforest automation was evaluating whether or not to pass that step.

Search element timeout per matching method

When using multiple fallback methods, the timing for how long our automation service will search for a match are as follows:

  • Screenshot + DOM + AI: Up to 60 seconds searching for a screenshot match + one check for a DOM match + one AI check. DOM and AI Search will only occur if a screenshot match is not found.
  • Screenshot + DOM: Up to 60 seconds searching for a screenshot match + one check for a DOM match.
  • Screenshot + AI: Up to 60 seconds searching for a screenshot match + one AI check.
  • DOM + AI: Up to 60 seconds searching for a DOM match + one AI check.
  • Screenshot only: Up to 60 seconds searching for a screenshot match.
  • DOM only: Up to 60 seconds searching for a DOM match.
  • AI only: One page search for an AI match.

FAQs

Can I disable DOM or AI matching entirely?

Yes. Individual matching methods can be disabled account wide from the global settings page.

Can I edit the selector or AI label?

Yes. Rainforest will generate values for the selector and the AI label if it can, but you can edit these as needed.

What if I don’t want to use a screenshot at all?

Just disable this method and use DOM or AI to search for the element you’d like to validate.

Can I change the order of matching fallbacks?

No, the fallback order will always be screenshot, then DOM, then AI.


If you have any questions, reach out to us at [email protected].