Test Length

There is no limit to the number of steps in a test, but the best tests are between 10 and 25 steps long.

There are exceptions, but testing for long processes should take the form of multiple, smaller tests against each individual part of the larger process. Shorter, more targeted tests are your secret to success.


1. Short tests are faster

Tests in a single run are performed by our pool of testers in parallel. This means one 5-step test and twenty 5-step tests will take approximately the same amount of time to complete. Conversely, a less focused, 25-step test will take much longer for testers to complete. From a time perspective, it's advantageous to keep each individual test as short and focused as possible.

2. Pinpoint accurate error reporting

If a large test fails part-way through, you'll be alerted to the failure where it occurred, but you will lose visibility to issues that may arise later in the test. Splitting longer processes down to the more granular components will ensure that you identify problems anywhere they might arise.

3. Easier test-case management

Focused tests allow you easily update your test suite when you make a change to a process. If you have a 5-part signup process and you make a change in part 3, it will be much simpler for the team to quickly update the test for only part 3, while the other tests can keep running.

Limiting Ambiguity and Vagueness

Within a test or test step:

  • Ambiguity is when several paths to a goal are available to the tester, and it's not clear to them which one you want them to take.
  • Vagueness is when a step or a test doesn't have enough context or detail to help guide the tester along an intended workflow.

Why should you limit ambiguity and vagueness?

Clear instructions make better tests and better tests lead to better test results. Clear instructions make it easier for testers to follow, which translates into better feedback returned to you.

Clear, concise instructions allow you to identify where and how bugs happen.

Steps should be concise and have a singular focus. If a clearly-written step fails, it is easier to see where exactly an error occurred. A step should tell a tester precisely what actions they need to do to pass it. Completing one step should unambiguously allow them to proceed to the next step in the flow.

Using a simple login flow as an example, a good log in step should look like this:

Action:
Log in by inputting in the field under Email with "example-rnfst@email.com" and the field under Password with "example.password1".

Question:
Did you see a green header that contained the message "You have logged in successfully"?

Here, the action identifies both where the tester should input the values to log in, and what those values should be.

Clear, concise steps are much easier to understand and follow.

Steps should be clear, so testers can better comprehend the instructions contained. The more they understand, the better they're able to make judgments as they go through a workflow. This means they're less likely to get stuck on steps that may seem obvious to you or fail to reach consensus.

If an error does occur, a clearly-written test allows you to recognize how the error occurred, not just where. Using the same example one more time:

Action:
Log in using email: "example-rnfst@email.com" and password: "example.password1".

Question:
Did you see a green header that contained the message "You have logged in successfully"?

It would've been just as valid to ask "Did it work?" as the question of this step. But, "Did it work?" doesn't provide much context on what or how the login functionality should function. The more you can explain in a clear and concise manner, the easier it is for testers to understand and execute the step correctly.

Clear, concise instructions enable testers to give useful feedback.

A well-written step provides clear acceptance criteria, so testers can make judgments as they go through the workflow. If they know the expected outcome, they can write their bug report based on concrete criteria. That way, you can pinpoint where the failure happened, what caused them to fail the step, and also why they had to fail it.

Using the same example from above:

Action:
Log in using email: "example-rnfst@email.com" and password: "example.password1".

Question:
Did you see a green header that contained the message "You have logged in successfully"?

Adding the phrase "You have logged in successfully" to the question is helpful because it gives testers an idea of the expected outcome of completing the step (i.e. 'If the instructions are correct, then I should be successfully logged in'). You could ask if they were logged in, but this extra bit of detail provides a reference point to assist the tester if they were to give feedback.

Limiting ambiguity and vagueness not only translates into more useful feedback, but it can also help tests pass quicker.

Did this answer your question?