Easier management of your test suite and results
In QA, not all tests are created equal. With Test Priority, you can set a priority for each test case in your suite to indicate the relative importance of that test flow to your app, and start to see results in order of priority (with P1 tests first).
What is Test Priority?
Test priority is a test-level attribute that allows you to define how critical a test is to your application using P1, P2, and P3 importance markers. We don't enforce any label beyond this, but like most other software, we'd reserve P1 for mission-critical tests that we would add to our smoke suite, while P3 can be used to indicate non-essential functionality that gets tested less frequently.
How to add Test Priority
In the app, there are several places to assign priority from:
- When you create a New Test, you must set the importance from the test creation flow. This is a required field, and we don't default to any value;
- When editing an individual test, you can use the dropdown in the navigation on the right hand side of the screen to set the priority. We don't store historical values for priority and will always consider the latest priority set;
- When requesting tests via Test Designer, you can specify how important each test returned to you should be when you upload the videos or introduce text outlines;
You can also use the CLI to add priority. When using the CLI, we will validate that a P1, P2 or P3 priority was added when you upload the tests.
Once you add a priority, you can't remove it - you can only change the priority of an existing test.
A few things to note
When you review results, you'll see tests listed in order of priority, with the most important ones first (instead of test name, the way it was before). If you have tests with no priority, those will show up last.
Questions? Feature requests? Send us an email at firstname.lastname@example.org to chat.