AI test generation

Overview

Rainforest can generate draft tests for you based on a prompt and the credentials. It's important to note that the test generated may not be perfectly set up to run without any modifications, but it can significantly save you time by doing most of the heavy lifting.

How it works:

1. Add credentials (if needed). If your test involves logging into your app, include the login details.
2. Write a prompt. Describe the goal(s) of the test case. Don't worry about capturing every detail, but use enough context so someone new to your app would understand what to do.
3. Generate the test. Our AI agents will spin up a virtual machine and break down your prompt into subtasks based on interacting with the app's UI, just like a human tester would. They’ll continue to add actions and UI elements until they’ve completed the task included in your prompt.
4. Review and refine. In a few minutes, you'll have a draft that you'll be able to add assertions, swap in test data or snippets, and set it to run—saving significant time compared to writing tests manually.

Limitations:

1. Test generation cannot call snippets: Generated tests don't use snippets nor pull in existing snippets. All test steps are flat. You can swap in snippets or create snippets after the draft is generated.
2. Test generation cannot use test data: Generated tests don't pull in any test data. You can replace any hard-coded data in the draft after it's generated.
3. Test generation cannot determine what should be a conditional: The generator writes steps for what appears on the virtual machine, so the agents won't be able to write steps for the counterfactual scenario that is not appearing at the time of generation. You can go back and make any of those steps conditional as needed.

Tips and tricks

1. Write instructions for the agents like you'd write for an assistant without much context: The agents are adept at deciphering common UI patterns, but if there are app-specific idiosyncrasies, it's best to include them in your prompt or site rules. This will make generation more reliable.
2. Use site rules, especially for basic information about your app: If you find yourself writing the same instructions or context in all your prompts, add these to the site-specific rules for generation. We'd specifically recommend giving the agents context about the purpose of your app (like you'd do for a QA assistant). This can only help with accuracy.
3. Be careful if writing many details into your prompt; the more detailed your instructions, the more room for error: The agents will take your instructions at face value; if you point them to click at a "sign in" button that doesn't exist, they'll get sidetracked trying to find it. If you're writing at a higher level (e.g., "Log into the application"), they'll have more flexibility to accomplish the task.