How to Categorize Failures

Learn how to use failure categories to provide more meaningful data to your team.

Failure categorization is essential to making sure you get the most information and understanding of your test suite’s health. It also ensures the proper action is taken when a test fails.

Failure Categories

Following are the six failure categories you can use.

BugAn issue that needs to be addressed. The feature or function doesn’t work as intended.
Known BugA bug that was previously identified.
Note: Use Bug to categorize the first instance of the issue and use Known Bug for any additional tests or run that resurface the same known bug.
Needs RewritingThe test needs to be updated (rewritten) to match the tested functionality.
Note: If the Test Writing Service is part of your plan, you can rewrite the test yourself or ask Rainforest to rewrite it for you. For more information, see Using the Test Writing Service.
Tester ErrorThe system doesn’t have a bug. Though the instructions were correct, the tester failed the test or didn’t follow the instructions.

NOTE: This option is only available if the test was run utilizing the Rainforest Tester Community.

See Working with Tester Community Results for more information.
My environmentThere was an issue with the system, such as connectivity or incorrect data.
Rainforest IssueThere was an issue with Rainforest such as VM issue.
OtherA catchall category for other items that might have caused the failure.

Accessing Failure Categorization

Failures are highlighted on several pages. You categorize failures either on the Run Summary page or the individual Test result page.

The Run Summary Page

The following steps show you how to categorize failures on the Run Summary page.
Updating a single test

  1. Click the Uncategorized button.
  2. Update Failure Status to Open.
  3. Select a failure Category as appropriate
  4. Update Assignee as appropriate.
  5. Provide comments as well to include important details about the failure categorization.

Updating a multiple tests

  1. Select the checkbox next to the tests you want to categorize.
  2. Select Update Failure Category from Bulk Actions, and select the appropriate option
  3. Use Bulk Actions to update any status or comments.
    Note: When bulk updating, the information will be applied to all selected tests.

The Test result Page

  1. From the test view, click on a previous failed result indicated by the red X icon.
  2. Click the Uncategorized button.
  3. Update Failure Status to Open.
  4. Select a failure Category as appropriate
  5. Update Assignee as appropriate.
  6. Provide comments as well to include important details about the failure categorization.

If you have any questions, reach out to us at [email protected].

Related Articles & Links