Image Matching Mode Choices in the Visual Editor

Learn about our image matching choices in the visual editor.

Overview

We have enhanced the way our virtual machines match elements to see more like humans and better tolerate small updates. For every screenshot in your tests, we will now give you a choice of three matching modes:

  1. Intelligent (the new default)
  2. Legacy
  3. Legacy with text fallback

Intelligent Matching Mode is able to see more like a human. This means that it will detect large & meaningful changes in your elements, but ignore the kinds of changes that most humans would ignore. Here are some things that it will ignore:

  • Small changes in color, font size, spacing, or rendering.
  • Appearance of text and mouse cursors over elements.
  • Unimportant differences in white space (padding, margins).

Why we’ve built the intelligent matching mode

We found that some of our customers’ runs would fail because the legacy matching mode couldn’t ignore small visual changes. For instance, notice the difference of the underline thickness between the elements on the left and right:

IF you’ve captured this elementAND your website or app shows this elementAND you’ve selected this Matching ModeTHEN the result would be thisBUT you expect
LegacyFAILPASS
IntelligentPASSPASS

Here are some situations to consider:
-If your tests fail due to slight visual changes, switching to intelligent matching mode is likely to help.
-If your element is currently matched using the legacy with text fallback mode, switching to intelligent matching mode will be the more strict option.
-Intelligent matching mode looks at text content, but requires both text to be the same and the appearance to be somewhat similar to pass - so if, for example, the font size changes by a very noticeable amount, the action might fail to find a match.

How to switch to Intelligent matching mode

  1. Navigate to an existing test
  2. On the left sidebar, click on the “element” part of an action
  3. Under “Matching Mode”, click “Change”
  4. Under “Select Matching Mode”, click “Intelligent”
  5. Click the back arrow (←) to confirm

Any new elements created will automatically use the new intelligent matching mode.

When to keep using the old matching modes

You might want to keep using the legacy matching mode if you want to detect minor visual changes. For instance:

  1. You want to make sure that some elements are laid out exactly as you intended them to be.
  2. Finding a button should fail if the padding changes slightly.

In contrast, you might want to use Legacy with text fallback when both of these apply:

  • you don’t really care about the appearance, only that specific text is visible on the page,
  • you’re confident the same text will not appear on the same page multiple times (e.g. in the window title bar and the main page itself).

How legacy matching modes work

When you add an action in the Visual Test Editor, Rainforest QA asks you to click and drag over the virtual machine to capture an interface element. After capturing an element you select a matching mode. In previous versions, Rainforest offered two matching modes: purely pixel based or pixel based with a text matching fallback (chosen by toggling "Allow Text Matching"). Whenever you run a test, Rainforest will try to match the element you’ve captured, against the element that shows up on your website. Based on the matching mode you've selected, and the closeness of the two screenshots, your action will result in a PASS or FAIL*. For instance:

IF you’ve captured this elementAND your website or app shows this elementAND you’ve selected this Matching ModeTHEN the result would be
LegacyPASS
LegacyFAIL
Legacy with text fallbackPASS

More about intelligent matching mode

Rainforest QA imitates seeing things like a human. This means that it will:

  • detect large changes in your elements,
  • ignore changes that most humans wouldn’t notice.

Here are some things that it will ignore:

  • small change in color, font size, spacing, or rendering,
  • appearance of text and mouse cursors over elements.

How it works

Our matching mode will tolerate the kinds of changes, that a person would find difficult to notice. Those small differences will result in a PASS, just as you’d expect if a human were to see it.

ReasonIF you’ve captured this elementAND your website or app shows this elementTHEN the result would be
Text decorationPASS
Font sizePASS
Font weightPASS
Image sizePASS
Spacing between elementsPASS
Text colorPASS
Background colorPASS
Mouse cursor shows up by accidentPASS
Text cursor that shows up by accidentPASS
Text that renders differently in Windows and macOSPASS

📘

Pro Tip

Even though our matching mode is tolerant to differences, we still recommend you regularly update your element screenshots so they’re always up-to-date with the latest state of your website or app, increasing test stability.

And here are examples that we’d still consider FAIL:

Case ChangeFAIL

If you have any questions, reach out to us at [email protected].