How Rainforest ensures the quality of your results

Delivering high-quality results is our number one priority. This is the core of our business, and we could write a whole book about how we do this. The following is a high-level guide to how we do Quality Control (QC).

Reputation

Every tester on Rainforest has a reputation score that is constantly adjusted based on their work. This is the foundation of our QC process. This score helps give Rainforest QA an idea of how well a tester is performing, and whether or not they are a right fit for our community.

Some of the things that impact a tester's reputation today (this is constantly changing):

  • their agreement or disagreement with high or low ranking testers
  • performance in our training and challenge tasks
  • their frequency of interaction with Rainforest
  • their response time to new Rainforest jobs

Training

Every single new Rainforest tester must go through a rigorous on-boarding module. This process allows the testers to fully understand the testing rules and expectations before accessing customer tests. After that, our testers continue to perform intermittent, automated training modules throughout their testing career to keep their skills sharp.

This training is multi-faceted, some of the things that we focus on with training are:

  • English comprehension and writing
  • ability to follow nuanced instructions
  • ability to correctly categorize bugs
  • ability to intuitively understand web interfaces

Rules

We have a set of rules and instructions that testers must follow in order to ensure that applications are tested in a standardized way, no matter who is performing the test. We highly encourage you to become familiar with our rules and what they mean for you if you haven’t already.

Results

We algorithmically review all results from testers in real time. This ensures high-quality results while maintaining the speed that we're famous for. Some of the things that the algorithm looks for today (this is constantly changing):

  • agreement with other testers (weighted by reputation)
  • past history of test
  • time taken by the tester
  • interactions by the tester with your website
  • interactions by the tester across the rest of their system during the test

Human-verified results

We also sample a number of testers jobs and review results on a daily basis through a tester peer review process, where a pool of highly knowledgeable and trusted testers review and flag regular tester work for review by Rainforest staff. Our peer reviewers help us review test results and provide feedback as to whether the work was accurate or not, as per the tester rules established.

Removing testers

Unfortunately, there are times when we need to remove tester based on their performance, or other factors. If a tester fails to pass retraining, their performance declines, or they are otherwise rude and unhelpful in their comments and feedback towards clients and Rainforest staff, our algorithms or our Community Team will take action.

When a tester is blocked, this means they will no longer be able to log into their Rainforest QA portal and complete customer jobs.

Please note: rating a tester does not mean they will be blocked automatically. This merely sends us a signal that their behavior needs to be further investigated before a decision is made. In fact, we encourage you to rate testers when they are both helpful and unhelpful, as it helps us improve the overall quality of our results.

Did this answer your question?