Generating Self-Healing Tests with AI blocks

Guide on how to generate Rainforest tests that self-heal.

Overview

Based on simple, plain-English prompts you provide, Rainforest uses generative AI to generate test steps and — upon test failure — to automatically regenerate those test steps in an attempt to self-heal. If regenerated test steps successfully self-heal, then Rainforest automatically updates your test to the new, passing state, saving you time on test maintenance.

To learn about best practices for self-healing with AI-generated test steps, visit this article.

Creating a test with self-healing steps

Write a prompt in plain English and Rainforest's AI will generate the associated steps under it, within the prompt's "block." The AI will generate the same type of steps that users can create manually within Rainforest.

Generating test steps based on a prompt

  • Click on the Prompt button OR click the ➕ button and select the AI block action from the list
  • Describe the action you wish to perform. For example:
    • "Add a product to the cart"
    • "Fill the form using dummy data"
    • "Visit the signup page and create an account"
  • Click the magic wand icon to generate steps under a prompt.
  • View the underlying steps being generated, as part of the AI block.

Using test data

  • When creating a prompt, type {{ into the prompt.
  • Select a variable from the popup.

Regenerating test steps

  • Click on the recycle icon to regenerate steps under a block.
  • The previously-generated steps will disappear and the new steps will appear as they're generated.

Stopping the generation of test steps

  • Click on the Stop Generating Steps button at the bottom of the sidebar OR the x icon next to the Generating Steps… message.

Previewing the test

Previewing a test with generated blocks of test steps from an AI block will not execute any test step generation — it will simply execute the test steps as they exist within the visual editor.

Previewing will ignore your plain-English prompts, treating them the same way as a Comment action. The steps generated based on your prompts, however, will be executed.

Preview a test containing a AI block

  • Select the steps you wish to preview.
  • Click on Preview.

Running the test

Running a test with AI blocks will follow this logic:

Running a test containing a AI block

  • Go to the Test page.
  • Run the test.

Working with results

For each type of test result, you can see a summary or explanation.

  • Successful: how many steps passed.
  • Self-healed: what changed in the test/steps between the attempts.
  • Failure: why the run failed.

Viewing results

  • Go to the Runs page.
  • View the icon indicating the result.
    • ✔: passed on first attempt, without retry/secondary-level matching (DOM, AI label).
    • ✔🟠: passed using either regular retry (not regenerated) or secondary-level matching (DOM, AI label).
    • ✔✨: passed on regenerated retry.
    • ❌: failed.

Viewing the run summary

  • Hover over the run result icon to view the run summary.

Viewing a successful run

  • Click on a run that has a green check mark.
  • View the underlying steps and the result.

Viewing a self-healed run

  • Click on a run that has a green check mark with purple sparkles.
  • View the underlying steps and the result.
    • Attempt 1: why it failed.
    • Attempt 2 (regenerated, self-healed & auto-updated): what changed in the test steps from attempt 1.

Viewing a failed run

  • Click on a run that has a red "X."
  • View the underlying steps and the result.
    • Attempt 1: why it failed.
    • Attempt 2: why it failed (might contain regenerated steps if the failed step in Attempt 1 was within a generated block - will not auto-update the test though).

Test history

You can view the test's version history and restore any version, including any updates resulting from regenerative self-healing.

Viewing the self-healed version of a test

  • Click on the view history icon.
  • View the changes made to the test by looking for Rainforest self-healing under the PERFORMED BY column.

Best Practices

Visit this article to learn more.


If you have any questions, reach out to us at [email protected].