How to Categorize Failures

Learn how to use failure categories to provide more meaningful data to your team.

Failure categorization is essential to making sure you get the most information and understanding of your test suite’s health. It also ensures the proper action is taken when a test fails.

Failure Categories

Following are the six failure categories you can use.

CategoryDescription
BugAn issue that needs to be addressed. The feature or function doesn’t work as intended.
Known BugA bug that was previously identified.
Note: Use Bug to categorize the first instance of the issue and use Known Bug for any additional tests or run that resurface the same known bug.
Needs RewritingThe test needs to be updated (rewritten) to match the tested functionality.
Note: If the Test Writing Service is part of your suite, you can rewrite the test yourself or ask Rainforest to rewrite it for you. For more information, see Using the Test Writing Service.
Tester ErrorThe system doesn’t have a bug. Though the instructions were correct, the tester failed the test or didn’t follow the instructions.

NOTE: This option is only available if the test was run utilizing the Rainforest Tester Community.

See Working with Tester Community Results for more information.
My environmentThere was an issue with the system, such as connectivity or incorrect data.
Rainforest IssueThere was an issue with Rainforest such as VM issue.
OtherA catchall category for other items that might have caused the failure.

Accessing Failure Categorization

Failures are highlighted on several pages. You categorize failures either on the Run Summary page or the individual Test Results page.

The Run Page

Selecting a test run from the Run page takes you to the Run Summary page, where you can categorize the failures.

23502350

Accessing the Run page.

The Run Summary Page

The following steps show you how to categorize failures on the Run Summary page.
Updating a single test

  1. Update Failure Status to Open.
  2. Update Failure Category as appropriate
  3. Update Assignee as appropriate.
  4. Provide comments as well by selecting the comment icon to right.
    Note:Once a comment has been provided, the comment icon turns green to easily highlight those issues with comments.

Updating a multiple tests

  1. Select the checkbox next to the tests you want to categorize.
  2. Select Update Failure Category from Bulk Actions, and select the appropriate option
  3. Use Bulk Actions to update any status or comments.
    Note: When bulk updating, the information will be applied to all selected tests.
23042304

Categorizing a failure via the summary page.

The Automation Services Test Results Page

After a test executes via the Automation Service, you can categorize any failures from the Test Results page.

23342334

The Automation Services Test Results page.

The Tester Community Test Results Page

After a test executes via the Tester Community, you can categorize any failures from the Test Results page.

23742374

The Tester Community Test Results page.

When selecting the failure type, you can provide additional information in the Failure Category field (up to 250 characters). Team members can view the information by hovering over the categorized issue. You can also alert a team member by using the Notify Team Member dropdown. Doing so sends an email about the issue.

To remove a failure category from a test, click the Failure Categorization dropdown, then select “Clear Category.”


If you have any questions, reach out to us at [email protected].


Related Articles & Links