We have enhanced the way our virtual machines match elements to see more like humans and better tolerate small updates. For every screenshot in your tests, we will now give you a choice of three matching modes:
- Intelligent (the new default)
- Legacy with text fallback
Intelligent Matching Mode is able to see more like a human. This means that it will detect large & meaningful changes in your elements, but ignore the kinds of changes that most humans would ignore. Here are some things that it will ignore:
- Small changes in color, font size, spacing, or rendering.
- Appearance of text and mouse cursors over elements.
- Unimportant differences in white space (padding, margins).
We found that some of our customers’ runs would fail because the legacy matching mode couldn’t ignore small visual changes. For instance, notice the difference of the underline thickness between the elements on the left and right:
|IF you’ve captured this element||AND your website or app shows this element||AND you’ve selected this Matching Mode||THEN the result would be this||BUT you expect|
Here are some situations to consider:
-If your tests fail due to slight visual changes, switching to intelligent matching mode is likely to help.
-If your element is currently matched using the legacy with text fallback mode, switching to intelligent matching mode will be the more strict option.
-Intelligent matching mode looks at text content, but requires both text to be the same and the appearance to be somewhat similar to pass - so if, for example, the font size changes by a very noticeable amount, the action might fail to find a match.
- Navigate to an existing test
- On the left sidebar, click on the “element” part of an action
- Under “Matching Mode”, click “Change”
- Under “Select Matching Mode”, click “Intelligent”
- Click the back arrow (←) to confirm
Any new elements created will automatically use the new intelligent matching mode.
You might want to keep using the legacy matching mode if you want to detect minor visual changes. For instance:
- You want to make sure that some elements are laid out exactly as you intended them to be.
- Finding a button should fail if the padding changes slightly.
In contrast, you might want to use Legacy with text fallback when both of these apply:
- you don’t really care about the appearance, only that specific text is visible on the page,
- you’re confident the same text will not appear on the same page multiple times (e.g. in the window title bar and the main page itself).
When you add an action in the Visual Test Editor, Rainforest QA asks you to click and drag over the virtual machine to capture an interface element. After capturing an element you select a matching mode. In previous versions, Rainforest offered two matching modes: purely pixel based or pixel based with a text matching fallback (chosen by toggling "Allow Text Matching"). Whenever you run a test, Rainforest will try to match the element you’ve captured, against the element that shows up on your website. Based on the matching mode you've selected, and the closeness of the two screenshots, your action will result in a PASS or FAIL*. For instance:
|IF you’ve captured this element||AND your website or app shows this element||AND you’ve selected this Matching Mode||THEN the result would be|
|Legacy with text fallback||PASS|
Rainforest QA imitates seeing things like a human. This means that it will:
- detect large changes in your elements,
- ignore changes that most humans wouldn’t notice.
Here are some things that it will ignore:
- small change in color, font size, spacing, or rendering,
- appearance of text and mouse cursors over elements.
Our matching mode will tolerate the kinds of changes, that a person would find difficult to notice. Those small differences will result in a PASS, just as you’d expect if a human were to see it.
|Reason||IF you’ve captured this element||AND your website or app shows this element||THEN the result would be|
|Spacing between elements||PASS|
|Mouse cursor shows up by accident||PASS|
|Text cursor that shows up by accident||PASS|
|Text that renders differently in Windows and macOS||PASS|
Even though our matching mode is tolerant to differences, we still recommend you regularly update your element screenshots so they’re always up-to-date with the latest state of your website or app, increasing test stability.
And here are examples that we’d still consider FAIL:
If you have any questions, reach out to us at [email protected].
Updated 3 months ago