Types of helpful feedback from testers
Testers have two ways to leave you feedback:
- Bug reports (i.e. No answers)
- Tester feedback
When a step can't be answered with Yes based on our rules and instructions, testers will select No If your instructions were well written, this is their opportunity to share a bug report to help you identify what went wrong.
The bug report
We ask that their report includes a constructive, helpful comment that explains why they had to fail the step. Please note that lengthy comments are not mandatory, and some testers may leave more detailed feedback than others.
Tester Feedback (aka Suggest Improvement)
Occasionally, you'll see speech bubble icons in results: these indicate extra feedback from testers.
A test that includes additional feedback from testers
This is an optional feature called "Suggest Improvement" (SI) for our testers. Testers may use it only if the answer to a question if yes — think of it as a "Yes, and …".
They might want to leave you a tip about how your instruction could be phrased better, or something else they noticed while testing. We encourage testers to use the SI feature in addition to answering the step with “YES”.
Tester feedback on a step result
Testers can use SI to report situations falling under the 5 categories listed below:
- This step can be broken into multiple steps: you may see this if you are asking them to do too many things in one step, and this is confusing;
- These instructions are hard to understand: you may see this if the testers bumped into some jargon they don’t quite understand, or phrasing that could be clearer;
- The action you want me to do is not fully described: you may see this if your instructions omit certain actions that are too far of a leap for testers to make, or you maybe forgot to add something to make them complete;
- There was an unexpected pop-up in this step: if they bump into a pop-up you didn’t account for in instructions, they think is benign (see Rule 3) but think you should know about it anyway;
- I see an additional problem that is not asked as part of the question: if they notice something else about your app, e.g. a typo in a part not mentioned in instructions.
While we recommend testers use this in certain situations, it is never mandatory. If the tester chooses not to leave a suggestion, they must follow your instruction to the best of their ability and answer Yes or No accordingly.
Should they choose to use it, we ask that their report includes a polite, helpful comment explaining how the situation could be improved. Unless the tester’s comment was rude or unhelpful, you should never give the tester a poor rating for using this feature at an appropriate time.
If you have any questions about testers or need to report a bad comment, please let our team know in as much detail as you can using email@example.com.