lecture 16 of 170 completed

Fixing flakiness across a whole suite

One flaky test is a bug. Fifty flaky tests is a process problem, and you fix it differently.

What you'll learn

  • How to fix flakiness across a whole suite rather than one test
  • Why measuring comes before fixing
  • How to agree a rule that stops it returning

You already know how to fix a single flaky test. This lecture is about the situation you will inherit: a suite where forty tests fail at random and everyone has given up.

You cannot debug forty tests one by one. You need a different approach.

Step 1: measure before you fix anything. Playwright's HTML report and JSON reporter both record which tests were retried. Collect that over a week and count failures per test.

Do not skip this. Without numbers, "the suite is flaky" is a feeling, and you cannot show progress to a manager who is asking why testing takes so long.

full course

Keep reading with the full course

The Foundations track and the first lessons in each framework are free. This lesson needs full course access. A course plan includes all lessons for Playwright, Cypress and Selenium.

See course plans →