What is a Regression Test?
A regression test is used to confirm that recent changes do not adversely affect (break) previously existing features. Ideally, a suite of 15 to 20 regression tests should provide a sufficient substantial level of coverage.
The Regression Test's Defining Characteristics:
- Granular - At the step level, each step of a regression test is meant to describe a single action or small group of actions (Log-in, Select an item, Add items to a cart) that a potential user could take during their interaction with the website or feature.
- Detailed - As regression tests are more complex than smoke tests, writing in an appropriate amount of detail into the steps can improve a tester's overall sense of how the test is supposed to function.
- Specific - A regression test is meant to provide coverage over a specific action-path. It has a defined starting point and guides a test toward definitive end-goal the tester is meant to reach.
When should I run regression tests?
Your regression tests should be run frequently throughout your development cycles and before any major releases. These tests are meant to provide coverage for any part of your app that can experience issues.
Why are regression tests important?
The goal of a regression test is that it is a reliable indicator of stability and consistency. The consistent passage of a regression test indicates an action-path is continually functioning as designed.
Like with smoke tests, the failure of a regression test should sound an alarm that something is preventing a user from using an action-path as expected. Failures could indicate a legitimate bug with the feature. It could also indicate that the test was unclear and resulted in inaccurate execution. about what the tester was actually supposed to do in their interaction. Either way, regression tests become a valuable pipeline of feedback that can be distilled into future improvements.