Overview

Rainforest QA is a no-code test automation platform built for CI/CD. To-date, more than ten thousand startups have used Rainforest’s UI tests to improve product quality, and we use Rainforest to test our own releases multiple times each day.

Evaluation

Here’s how we recommend you evaluate Rainforest as part of your CI/CD pipeline to see if it’s a good fit for your team:

1) Create your first automated test

1. Start by creating an automated test that covers one of your most common user flows (e.g., signup or login flows).
2. Add your first test to a Run Group so that it can be kicked off using the CLI.

2) Manually trigger a test run

Manually triggering a run using the CLI can help you find and test the arguments you want to use in your CI config. Once you've got your CLI invocation starting runs like you want, simply copy the command over to your CI tool.

1. Install the Rainforest CLI.
2. Trigger a run using the arguments shown in the right-side column of the Run Group page:

3) Configure your CI Tool

1. Find your API token in the Settings > Integrations page of your Rainforest Account.
2. Store your Rainforest API token in your CI tool’s environment.
   • The simplest approach is to name the environment variable RAINFOREST_API_TOKEN, since that’s where the CLI/Orb will look by default.

📘

Tip

For users of the GitHub Action, you’ll want to use a GitHub secret instead. This is documented here.

3. Configure your CI tool to run Rainforest tests as part of your CI/CD pipeline.
   • If you use CircleCI, we have a CircleCI Orb for straightforward integration.
   • If you use GitHub Actions, you can integrate with our GitHub Action.
   • You can use the CLI to connect to any CI tool. Here are guides to a couple of popular tools:
     • Jenkins
     • Travis CI
4. Set your default CLI execution method to Automation.
   • Update Account Information
   • Individual tests can be overridden with:
     --execution-method automation
--execution-method crowd
5. Test your integration by triggering your CI tool to run. You can do this by committing a change and pushing it to your git repository.

4) Expand your test coverage

Next, to make sure the most critical flows in your app are covered, create a handful of smoke tests and add them to your Run Group(s).

Once you’ve created your initial smoke suite, here are a few tips for expanding your test coverage in a way that’ll protect the quality of your app while also keeping your releases moving fast:

• Do cover all the important user flows of your application.
• Do add your tests to Run Group(s).
• Don’t add more tests than you’re willing to maintain.
• Don't add tests that take too long to run, since you want your CI to be fast.
• Don’t test things you wouldn’t immediately fix if they broke.

Reference

Where Rainforest fits into CI/CD:
Not all CI workflows will look exactly like this diagram (and that's ok!), but it should help give you a better understanding of how we think Rainforest should fit in.

Typically your CI process will build and deploy your software to some QA or pre-production environment. From there, it'll run your Rainforest tests, and if they're successful, deploy your software to your production environment. If the Rainforest tests don't pass, you'll do some triage, fix whatever was wrong, and then restart CI back from the beginning.

Sample CI/CD Workflow:

More guidance on expanding test coverage:
The Snowplow Strategy: Improve Automation Test Coverage in Five Steps

If you have any questions, reach out to us at [email protected].