Using the CLI for Continuous Integration

Overview

Rainforest makes it easy to integrate testing into your release process. Configure virtually any CI tool to run Rainforest tests as part of your CI/CD pipeline. If you are looking for more information on running tests from the CLI please visit our page on executing tests from the CLI. For more information, see:

If you use CircleCI, a hosted continuous integration (CI) server, our CircleCI Orb allows for an easy, out-of-the-box integration.

Likewise, if you use GitHub Actions for your CI/CD, we offer a GitHub Action for easy integration there as well.

CI Integration with the Rainforest CLI

To integrate Rainforest with any CI server or deployment tool, we wrote a command-line interface that triggers a run of your tests from any script. For more information, see How to Install the CLI. You can find your API token in the Settings > Integrations page of your Rainforest Account.

Run a specific run group using the following command:

rainforest run --run-group <run_group_id>

Note: This can also be copied from Settings section of the run group page with the corresponding run group ID. Additionally this run will follow the run group settings you have set, if you would like set a different --execution-method be sure to specify in the command.
This would look like:

rainforest run --run-group <run_group_id> --execution-method automation

Run all your tests using the following command:

rainforest run --token <YOUR API TOKEN> --fail-fast --description "CI automatic run" all

The --fail-fast option makes the Rainforest tool return immediately after the first failure.
The --description option adds an arbitrary description to the run.
For a comprehensive list of options, refer to our open-source project.

To view examples of how to configure standard CI tools to automatically run your test suite, see our CI Sample Repository.

Git Triggers with Rainforest CLI

The Rainforest CLI supports triggering based on the contents of a git commit message. This is useful when Rainforest is integrated into your CI server, but you want more control over what is tested.

To enable this feature, add --git-trigger to the command line options you pass to Rainforest. With this flag enabled, the CLI client does nothing unless the last commit message for your project contains @rainforest and one or more hashtags. Here’s an example:

"Fixed the login button" is the last commit.
Instead of triggering rainforest "Fixed the login button. @rainforest #login", we start a rainforest run with anything tagged login.

Use git-trigger to enable teams to test feature branches for speed before your work is merged into a release branch. For more details on how to get started with this workflow, contact our Support team.

CI Integration with CircleCI

For the whole picture, we encourage you to consult CircleCI’s Getting Started documentation. If CircleCI is already set up, check out our CircleCI Orb to easily integrate Rainforest tests into your CI flow.

The config.yml File

The primary way to configure the CircleCI tool is to store a file in your git repository. You can see the sample configuration for our project here.

Notice that instead of entering the Rainforest API key directly in the config.yml file, we use environment-specific test data. We do this for security reasons. We don’t want to expose sensitive information to any reader of the file, especially when the repository is public. For more information, see Using Test Data.

Configure test data in your project’s Test Data settings. You can find your Rainforest API token by navigating to your Integrations settings.

Create a RAINFOREST_TOKEN environment variable in CircleCI corresponding to your test data, and you’re done. For more information, see CircleCI’s Using Environment Variables.

Our project is configured to provide a continuous delivery flow by default. Every time code is pushed to the develop branch, CircleCI:

Runs the automated tests.
Deploys the develop branch to a staging server.
Runs the Rainforest tests using the CLI on the staging server.
Once the Rainforest tests are successful, CircleCI pushes the develop branch to the master branch.
The previous step triggers another CI build, which pushes the code to production.

If any step fails, the build fails. This prevents the deployment of broken code to your users.

Testing Your Setup

You can test your setup by committing a change to your develop branch and pushing it to your git repository. CircleCI picks up the change and deploys your code. With a new continuous delivery flow and QA process, you can move fast while not breaking anything.

Note: If you don’t see an API token in your Rainforest settings, click the Rotate API Token button to generate a new one.

If you have any questions, reach out to us at [email protected].