Overview

Failure categorization is easy and essential to ensure that you get the most information and understanding of the health of your test suite.  It also ensures that proper action is being taken when a test does fail.  Failure categorization is also used to populate the weekly QA Metrics Report.   

Failure categorization is is highlighted on several pages.  
From the Dashboard and Runs page you will be advanced to the Runs Summary Page where you can actively categorize the failure
The Runs Summary page and the Test Results page you can directly categorize failures.  

Accessing Failure Categorization 

Failure Categorization is highlighted on several pages

  • Dashboard Page
  • Runs Page
  • Runs Summary Page
  • Test Results Page  

Dashboard Page

Advances you the Runs Page with a filter to show only failed tests / runs.  
Selection of any line will advance you to the Run Summary Page where you can actively categorize any failures.

Runs Page

Selection of any line will advance you to the Run Summary Page where you can actively categorize any failures.

Runs Summary Page

Actively categorize failures by selecting Needs Categorization option. 

Test Results Page

Actively categorize failures by selecting Categorize Failure dropdown

Failure Categorization Categories and actions

Categorization on the Runs Summary page and the Test Results page work similarly.

  1. Select Needs Categorization 
  2. Select the proper category

Upon selecting the issue type, you may provide additional information in the Failure Category dialog box (up to 250 characters).  This information will be visible to any teammate by hovering over the categorized issue.  You may also Notify a team member from the dropdown selected.  This teammate will receive an email about the issue.

NOTE:  if an issue is re-selected and SAVED, ALL prior Failure Categorization dialog box information will now not be visible. 

As issues are categorized for a run, the toucan will fill with more color and the Failure Category Breakdown section will update how the failures were categorized.  Just another way Rainforest makes QA enjoyable.

Hovering over the issue display the detail information and the team member that was notified (if provided).  

Failure Categories

The following are the five failure categories available. 

  • Bug - an  issue that needs to be fixed in the system being tested
  • Needs Refactor - the test itself needs to be updated (rewritten) to match new functionality on the system being tested.  
    If Test Writing Engine (Test Designer) is part of your suite, you may select to rewrite the test yourself or have Rainforest rewrite it for you.
  • Tester Error - the system doesn't have a bug, the instructions were correct, however the tester incorrectly failed the test for some reason
  • System Issues - there was an issue with the system such as login credentials, connectivity, incorrect data.  Additionally you may also select if the issue is with your own environment or if you believe it is a Rainforest VM or device issue
  • Other - catchall for any other items that could have caused the failure

Did this answer your question?