The Rainforest regression testing product allows you to write scripted test cases and execute them continuously and on-demand by humans (our Rainforest Tester Community) or machines (our automation bots).
We support two different test types - Rainforest Automation Language (RFA) and Plain English. There is no need to choose just one - Rainforest allows you to combine both, and mix-and-match tests in your suites, for a balanced testing strategy that takes advantage of human judgment/validation where needed, and the speed/cost savings of automation when it's not.
In this article, we'll break down the differences between the two to help you determine parts of your test suites are best suited for Rainforest Automation vs. Plain English.
A quick word - while this article will help you understand the differences between the two, we highly recommend writing and running a simple test in each language to get a feel for the nuances.
Summary: Plain English Test vs. Rainforest Automation Tests
1. Plain English:
Plain English tests are comprised of steps in action/question format. In the action portion of each step, you tell testers what to do and specify how to interact with your application; in the question portion of the step, you ask a Yes/No question to verify expected behavior. Once written, Plain English tests can only be run by our Tester Community.
Example of a Plain English test below:
2. Rainforest Automation Language (RFA):
RFA tests are no-code and written in structured, natural language. In this language, you define actions that you want the automation bot to complete (like click, paste, navigate to a new browser window, drag&drop, hover, scroll, etc) and the targets (screenshots of the UI elements) you want the actions completed on.
Rainforest Automation tests are flexible in terms of how they can be executed. A test written in RFA can be run by our Automation bot, or be compressed into natural language and run by our Tester Community, as illustrated in the gif below.
Below, we see how a test written in Rainforest Automation Language can be toggled to the "Plain English" version and run by Rainforest Automation or our tester community.
To visualize it in another way, the image below displays the two test types and execution types. As we can see, tests written in Rainforest Automation can be run by Automation or by our Tester Community, whereas tests written in Plain English can *only* be run by our tester community.
Rainforest Automation vs. Plain English: a side-by-side comparison
The table below illustrates the key differences between Plain English tests vs. Rainforest Automation tests in a side-by-side comparison; key differentiators are highlighted in green.
Rainforest Automation Tests (when executed by Automation):
- Fast, cheap execution
- Quick and easy to write and maintain
- No-code, so can be written and maintained by folks without technical expertise!
- Highly visual - tests what your users see, not what their computers see
- You can also execute them with our Tester Community
- Web-only at the moment (no mobile support)
- When run by automation, tests can only be run against the browser they were written for, as the small changes in rendering between different browsers can cause automation tests to fail
- Can be difficult for complex tests that involve dynamic UIs
Best Suited for:
- Scaling costs - any subset of tests that's run very frequently it makes sense to automate (smoke tests, tests run as part of CI)
- Tests that cover stable features
When would I want to run Automation tests against the Tester Community?
- When you want to run the test against browsers *other* than the one the test was written against
- At times when human feedback is important to you
Plain English Tests executed by our Tester Community:
- written in free-form natural language
- mobile web and native mobile app testing supported (as well as web)
- tests are browser-agnostic; can run a test against any (or multiple) browsers or mobile device combinations at once
- can cover extremely complex test cases - tests that require human judgment, or contain dynamic data / cover dynamic UIs
- Slightly slower and more expensive to run than Automation tests, as humans are involved
Best suited for:
- tests that require human judgment or human interpretation
- tests that cover unstable features or areas of your application that change frequently
Tests are fast to write, fast to maintain, and less brittle than automation. Plain English tests run against our Tester Community are ideal for helping you scale the kind of testing that requires human judgment or validation (i.e. "does this look like a dog?" "does the formatting of the page look right?")
Tests that must be intentionally vague, free-form, or require a level of human judgment or interpretation should be written in Plain English, and run against our tester crowd. Unstable features or areas of your application where the UI frequently changes are also a good fit for Plain English, as the testers can be more flexible in their interpretation of a test than an automation bot can.
Tests are fast and cheap to run, and relatively quick to write and maintain, but can be more brittle than Plain English tests run by the Tester Community. This is not unique to Rainforest Automation - any test automation will be more brittle than human testing, as tests must be written with a level of specificity and detail.
Automation is a good fit for web-only tests that you plan to run very often (smoke tests or tests that run as part of your CI process) against a single browser, as you'll benefit from the quick and cheap execution. When a layer of human review is important to you, you can run your Automation tests against our human tester crowd as well, across multiple browsers.
Write a test in each language to get a feel for the differences yourself!