One of the key reasons why people buy Rainforest is for the benefit of a crowd-powered QA testing solution. So here’s a conundrum: for non-automated tests, how can you ensure speed given test execution against a crowd of folks who have to take things step-by-step and be as thorough as they can be?
We’re often asked, “How can I get my tests to run faster?”
In a world where time is of the essence, here are some tips and tricks to speed things up a bit, whether you’re running one or multiple tests:
QA strategy tips
- Try running on fewer platforms. If speed is paramount, must you run a test on 4 browsers? Or could you narrow down two platforms that are crucial?
- Make the tests short and modular: under 25 steps. The shorter and more modular a test is, the faster and more accurate the test results will be. Each test should cover one feature or functionality, and we recommend you avoid long end-to-end tests. Whenever you add a step, not only does this increase execution time, but it also adds a potential new point of failure.
- Break your steps up, and don’t cram too much in them. Our testers have to read, process, execute, and remember what you’ve asked them to do. A quick succession of buttons to click or fields to enter is fine. Asking testers to complete a 30-field form in one step is not: testers want to be accurate, which only adds to the time it takes them to complete your step.
- Combine steps where it makes sense to really get to the point of the test. Think critically about how granular each step is, and what you’re really checking for. It also makes sense to structure your question so that it clearly links to the actions described in the step.
- Try running on a popular browser. Runs using uncommon VM types can also be slower, as they are not in high demand. If it’s not integral to your testing strategy, try browsers like chrome_1440_900.
- Check if there’s a concurrent tester limit on your account. This is the max number of testers that can access your product at the same time. It’s used to throttle traffic on limited-capacity servers, and may slow down run times. You can find it in Settings > Global Settings
- Take tester feedback into account. When they pass a step, testers also have the option to leave comments about other things they noticed on the page or confusing instructions, to help you refactor instructions. If they don’t understand what you want, they will dwell on that step for a long time.
- Examine tester videos to see how much time they spend on each step. Our videos have markers for step completion and can be sped up to 5x the original speed if you don’t have a lot of time. If they spend 10 minutes on a step, perhaps things aren’t clear enough from the instructions.
- Preview the test when writing or refactoring: do you go through it yourself in a reasonable amount of time? The tester will experience it just like you, except less familiar with your application since they will encounter it for the first time.
- Add screenshots or GIFs sparingly. Every step shouldn’t be a screenshot, but if your UI is busy or you want testers to find something buried deep, strategically placed screenshots help. It makes it easier for to testers understand how to execute complicated actions OR show them what elements of the page they should be focused on so that they don’t scour themselves for a long time.
Test writing and account seeding
- Keep the instructions short, and follow the test writing helpers in-app. They are there for a reason and will guide you so you don’t use quotes when not necessary to the test flow (they can increase verification time if you need testers to look for an exact match), and they encourage you to keep your instructions short.
- Stay away from using internal company jargon. Stick to language that describes the functionality as it appears to end users.
- Make use of Click to Copy. Click to copy works on text and links, and can be added from the app or the CLI. When you use it, testers will only have to click to copy the words, phrases or links and paste them into the terminal, which speeds things up.
- Make sure you have enough reusable step variables. If you use this feature, make sure you have enough of them, or disable that option so you don’t find yourself unable to start the run on time.
- Make sure that all the necessary states are already seeded when testers start working. Whatever this means for your app, it’s much faster (for instance) if you have an item that’s already marked as “ready to ship” instead of getting the testers to go through steps to prepare it. If you invest in proper seeded states upfront, you’ll see a payoff later on when our crowd of testers executes your steps because there’s less for them to do.
We hope this gave you ideas you hadn’t thought about and helped speed up run times.
Helpful? Let us know by reviewing this article below.