How our Algorithms determine how many testers are needed
With our Simple Vote Multi-fail algorithm, a minimum of 3 testers are assigned to each test per browser.
For example, running 1 test on 1 browser will send at least 3 testers. Running 1 test against 3 browsers will send 9 testers. Running 3 tests against 3 browsers will have a minimum of 27 testers, and so on.
If our algorithm determines that additional testers are needed to reach a consensus, or testers abandoned a test in progress for whatever reason, it will recruit more testers to deliver a reliable result for the test.
If you are setting up variables for your tests, a good rule of thumb is to count:
Number of tests containing variable * number of browsers testing across * 6 (testers).
This is the most basic method of Quality Control we do — if all testers reach a consensus on the outcome of the test, then we return the result to you.
To learn more, check out how we algorithmically evaluate results!