lecture 15 of 160 completed

Fixing flakiness across a whole suite

One flaky test is a bug. Fifty flaky tests is a process problem, and you fix it differently.

What you'll learn

  • How to fix flakiness across a whole suite rather than one test
  • Why measuring comes before fixing
  • How to agree a rule that stops it returning

You already know how to fix a single flaky test. This lecture is about the situation you will inherit: a suite where forty tests fail at random and everyone has given up. Older Selenium suites are where this problem is most common, so you will probably meet it early.

You cannot debug forty tests one by one. You need a different approach.

Step 1: measure before you fix anything. Selenium gives you nothing here, so build it yourself. Save each run's results to a file and count failures per test over a week. A simple spreadsheet is enough to start.

members only

Keep reading with the full course

The Foundations track and the first 5 lectures of each framework are free. This lecture is part of the paid course, which unlocks every lecture, all three frameworks, and downloads.

See the options →