Overview

Rainforest AI now automatically suggests steps to update any failing test, even if the failed step isn’t within an AI block.

How It Works

During a test run, when an automated test step fails:

If Rainforest AI determines the failure isn’t the result of a bug in your app, it’ll override the failing step with new steps to get your test back on track. The AI will generate steps until it determines it has achieved the goal of the original failed step.
If Rainforest AI’s new steps pass, the test will be marked with a purple label to indicate a potential fix is ready for your review.

When the run is complete, you can review the potential fix and decide whether to apply the fix to your test so it passes next time.

How Does This Work with AI Blocks?

Wherever possible, you should use AI blocks to create test steps. The context from your plain English prompts in AI blocks gives Rainforest AI the best understanding of your intent, allowing it to heal your failing tests appropriately without any intervention from you.

Moreover, AI blocks that self heal will pass by default, so you can save time compared with reviewing and implementing potential fixes from the AI.

	Self healing with AI blocks	Self healing of manually created steps
Are edits to your current tests required?	Yes	No
What happens when a test step fails?	The first attempt fails. On the second attempt, Rainforest AI will regenerate any steps with an AI block at or past the point of failure.	After a step fails, Rainforest AI immediately intervenes and attempts to achieve the original intent of the test. It will try to generate new actions to work through or around an issue that doesn’t seem like a bug.
Are changes incorporated immediately?	If the self-healed test passes, then a new version of the test is saved.	No. There is an option to apply the potential fix to your test.
Does the test pass if successfully healed?	Yes. The test will be marked with a green checkmark with purple sparkles.	No, the test will still be considered a failure.

FAQs

Can I turn off self healing outside of AI blocks?

Please message us via the in-app chat or at [email protected], and we’ll turn the feature off for you.

Does Rainforest AI attempt to fix any failing step?

As noted above, Rainforest AI doesn't attempt to recover any scenarios that appear to be actual bugs. If there's ambiguity about whether an issue is a bug or not, it will not attempt to recover.
Rainforest AI won't attempt to recover that last step in a test because there's no exit condition; typically AI attempts to get your test back on track but successfully getting to the step following the failed step.
Rainforest AI doesn't have the ability to generate actions with masks, text matching, drag and drop steps, and click variations (i.e. double click, click right-mouse, click and hold)

I’m not seeing the option to apply the fix for my tests. Why is that?

There could be a couple reasons for this. You’ll only see the option to apply a fix if:

Our attempt to self-heal wasn't successful. Rainforest AI might not have been able to recover from the failed step.
Self healing was successful. I.e., the AI successfully worked around the failure to progress to the next step in your test.
The test has not been edited since the run with the potential fix.

How do self healing fixes apply to Snippets?

If a parent test fails on a step that’s in a snippet, you can apply a fix to a step. This will impact any other tests that include that snippet.

It looks like Rainforest AI was fixing my test then stopped completing the self-heal. Why is that?

We don’t want to work around actual issues that really should require some intervention from you, so the AI is limited to a limited number of actions.

How does self healing work with retries?

Self healing will be attempted each time the test is run, so if you have multiple retries, we’ll try to work around any failure that comes up in any job.

Does self healing impact conditionals blocks?

A conditional parent step (i.e., the if statement), can be true or false, but can’t fail. So, self healing will never be triggered for the parent step in a conditional. If any child steps under the parent fail, Rainforest will attempt to heal those steps.

If you have any questions, reach out to us at [email protected].