Generating Self-Healing Tests

Guide on how to generate Rainforest tests that self-heal.

Overview

Based on simple, plain-English prompts you provide, Rainforest uses generative AI to generate test steps and — upon test failure — to automatically regenerate those test steps in an attempt to self-heal. If regenerated test steps successfully self-heal, then Rainforest automatically updates your test to the new, passing state, saving you time on test maintenance.

To learn about best practices for self-healing with AI-generated test steps, visit this article.

Creating a test with self-healing steps

Write a prompt in plain English, and Rainforest's AI will generate the associated steps under it.

Generating test steps based on a prompt

  • Click on the Describe steps to add OR click the ➕ button.
  • Describe the action you wish to perform. For example:
    • "Add a product to the cart"
    • "Fill the form using dummy data"
    • "Visit the signup page and create an account"
  • Press the Enter key to generate test steps.
  • View the underlying steps being generated, as part of the self-healing snippet.

Using test data

  • When creating a prompt, type {{ into the prompt.
  • Select a variable from the popup.

Regenerating test steps

  • Click on the 3-dot icon next to the snippet name and select Regenerate steps from prompt
  • The previously generated steps will disappear, and the new steps will appear when they're generated.

Stopping the generation of test steps

  • Click on the Stop generating steps button at the top of the sidebar

Previewing the test

Previewing a test with a self-healing snippet will not execute any test step generation — it will simply execute the test steps as they exist within the visual editor.

Previewing will ignore your plain-English prompts, treating them the same way as a Comment action. The steps generated based on your prompts, however, will be executed.

Preview a test containing a self-healing snippet

  • Select the steps you wish to preview.
  • Click on Preview.

Running the test

Running a test with self-healing snippets will follow this logic:


Working with results

For each type of test result, you can see a summary or explanation.

  • Successful: how many steps passed.
  • Self-healed: what changed in the test/steps between the attempts.
  • Failure: why the run failed.

Viewing results

  • Go to the Runs page.
  • View the icon indicating the result.
    • ✔: passed on first attempt, without retry/secondary-level matching (DOM, AI label).
    • ✔🟠: passed using either regular retry (not regenerated) or secondary-level matching (DOM, AI label).
    • ✔✨: passed on regenerated retry.
    • ❌: failed.

Viewing the run summary

  • Hover over the run result icon to view the run summary.

Viewing a successful run

  • Click on a run that has a green check mark.
  • View the underlying steps and the result.

Viewing a self-healed run

  • Click on a run that has a green check mark with purple sparkles.
  • View the underlying steps and the result.
    • Attempt 1: why it failed.
    • Attempt 2 (regenerated, self-healed & auto-updated): what changed in the test steps from attempt 1.

Viewing a failed run

  • Click on a run that has a red "X."
  • View the underlying steps and the result.
    • Attempt 1: why it failed.
    • Attempt 2: why it failed (might contain regenerated steps if the failed step in Attempt 1 was within a generated snippet - will not auto-update the test though).

Test history

You can view the test's version history and restore any version, including any updates resulting from regenerative self-healing.

Viewing the self-healed version of a test

  • Click on the view history icon.
  • View the changes made to the test by looking for Rainforest self-healing under the PERFORMED BY column.

Best Practices

Visit this article to learn more.


If you have any questions, reach out to us at [email protected].