How to Categorize Failures
Learn how to use failure categories to provide more meaningful data to your team.
Failure categorization is essential to making sure you get the most information and understanding of your test suite’s health. It also ensures the proper action is taken when a test fails.
Failure Categories
Following are the six failure categories you can use.
| Category | Description |
|---|---|
| Bug | An issue that needs to be addressed. The feature or function doesn’t work as intended. |
| Known Bug | A bug that was previously identified. Note: Use Bug to categorize the first instance of the issue and use Known Bug for any additional tests or run that resurface the same known bug. |
| Needs Rewriting | The test needs to be updated (rewritten) to match the tested functionality. Note: If the Test Writing Service is part of your suite, you can rewrite the test yourself or ask Rainforest to rewrite it for you. For more information, see Using the Test Writing Service. |
| Tester Error | The system doesn’t have a bug. Though the instructions were correct, the tester failed the test or didn’t follow the instructions. NOTE: This option is only available if the test was run utilizing the Rainforest Tester Community. See Working with Tester Community Results for more information. |
| My environment | There was an issue with the system, such as connectivity or incorrect data. |
| Rainforest Issue | There was an issue with Rainforest such as VM issue. |
| Other | A catchall category for other items that might have caused the failure. |
Accessing Failure Categorization
Failures are highlighted on several pages. You categorize failures either on the Run Summary page or the individual Test Results page.
The Run Page
Selecting a test run from the Run page takes you to the Run Summary page, where you can categorize the failures.
Accessing the Run page.
The Run Summary Page
The following steps show you how to categorize failures on the Run Summary page.
Updating a single test
- Update Failure Status to Open.
- Update Failure Category as appropriate
- Update Assignee as appropriate.
- Provide comments as well by selecting the comment icon to right.
Note:Once a comment has been provided, the comment icon turns green to easily highlight those issues with comments.
Updating a multiple tests
- Select the checkbox next to the tests you want to categorize.
- Select Update Failure Category from Bulk Actions, and select the appropriate option
- Use Bulk Actions to update any status or comments.
Note: When bulk updating, the information will be applied to all selected tests.
Categorizing a failure via the summary page.
The Automation Services Test Results Page
After a test executes via the Automation Service, you can categorize any failures from the Test Results page.
The Automation Services Test Results page.
The Tester Community Test Results Page
After a test executes via the Tester Community, you can categorize any failures from the Test Results page.
The Tester Community Test Results page.
When selecting the failure type, you can provide additional information in the Failure Category field (up to 250 characters). Team members can view the information by hovering over the categorized issue. You can also alert a team member by using the Notify Team Member dropdown. Doing so sends an email about the issue.
To remove a failure category from a test, click the Failure Categorization dropdown, then select “Clear Category.”
If you have any questions, reach out to us at [email protected].
Updated about 1 year ago