lecture 16 of 170 completed

Fixing flakiness across a whole suite

One flaky test is a bug. Fifty flaky tests is a process problem, and you fix it differently.

What you'll learn

  • How to fix flakiness across a whole suite rather than one test
  • Why measuring comes before fixing
  • How to agree a rule that stops it returning

You already know how to fix a single flaky test. This lecture is about the situation you will inherit: a suite where forty tests fail at random and everyone has given up.

You cannot debug forty tests one by one. You need a different approach.

Step 1: measure before you fix anything. Playwright's HTML report and JSON reporter both record which tests were retried. Collect that over a week and count failures per test.

Do not skip this. Without numbers, "the suite is flaky" is a feeling, and you cannot show progress to a manager who is asking why testing takes so long.

members only

Keep reading with the full course

The Foundations track and the first 5 lectures of each framework are free. This lecture is part of the paid course, which unlocks every lecture, all three frameworks, and downloads.

See the options →