Failure categorization is essential to making sure you get the most information and understanding of your test suite’s health. It also ensures the proper action is taken when a test fails. Failure categorization is also used to populate the Weekly QA Visibility Report and report issues to the Rainforest Support team.
Following are the six failure categories you can use.
An issue that needs to be addressed. The feature or function doesn’t work as intended.
A bug that was previously identified.
The test needs to be updated (rewritten) to match the tested functionality.
The system doesn’t have a bug. Though the instructions were correct, the tester failed the test or didn’t follow the instructions.
There was an issue with the system, such as connectivity or incorrect data. You can select whether the problem is in your environment or in Rainforest’s VM.
A catchall category for other items that might have caused the failure.
Failures are highlighted on several pages. You categorize failures either on the Run Summary page or the individual Test Results page.
The Dashboard page shows issues categorized as Bug or Needs Refactor over the last seven days. Selecting a test from the list takes you to a page where you can review and update.
Clicking Go Categorize takes you to the Run Results page, filtered to show failed tests. From there, you can categorize your tests.
Selecting a test run from the Run Results page takes you to the Run Results Summary page, where you can categorize the failures.
The following steps show you how to categorize failures on the Run Results Summary page.
- Click the checkbox next to the tests you want to categorize.
Failure categorization is only available for tests that have failed.
- Click the Categorize Failure dropdown.
- Choose the appropriate failure category.
Note: When you click multiple checkboxes simultaneously, the failure category, comments, and assignments are applied to all selected tests.
After a test executes via the Automation Service, you can categorize any failures from the Test Results page.
After a test executes via the Tester Community, you can categorize any failures from the Test Results page.
When selecting the failure type, you can provide additional information in the Failure Category field (up to 250 characters). Team members can view the information by hovering over the categorized issue. You can also alert a team member by using the Notify Team Member dropdown. Doing so sends an email about the issue.
To remove a failure category from a test, click the Failure Categorization dropdown, then select “Clear Category.”
If you have any questions, reach out to us at [email protected] or via the chat bubble below.
Updated about a month ago