lecture 10 of 160 completed

Keeping three thousand tests alive

Large suites die from entropy, not from bugs. Selector governance, dead test detection, and the decision to delete.

Every large suite I have inherited had the same three diseases: selectors nobody governs, tests nobody can justify, and a refactor everyone is afraid to start. None of them are bugs. They are entropy, and entropy needs a process rather than a fix.

Disease one: selector drift.

Four teams, four conventions. Some tests use data-testid, some use CSS classes, one file uses XPath through a plugin someone added in 2023. A UI redesign breaks two hundred tests and nobody can estimate the repair.

The cure is a written policy plus a lint rule. The policy is one paragraph: test IDs are required on interactive elements, named data-testid, in kebab-case, added by the developer writing the component. The lint rule is what makes it real:

// a custom ESLint rule, or a simple grep in CI
// fail the build if a spec contains a class selector or nth-child

A policy without enforcement lasts about six weeks. I have watched it happen more than once.

Getting developers to add test IDs is a negotiation, not a technical problem, and it is worth naming the argument that works: test IDs make their debugging easier too, and a missing one costs QA hours per instance. Framing it as a shared cost gets further than framing it as a QA requirement.

Disease two: tests nobody can justify.

A suite accumulates tests. It rarely sheds them. After three years nobody knows which tests cover what, whether two tests check the same thing, or whether a test still corresponds to a feature that exists.

Find the dead ones. Three signals, cheap to gather:

  • Never failed in a year. Not proof of uselessness, but a strong prompt. Combined with a low-traffic screen it is usually a delete.
  • Fails constantly and gets skipped. A .skip older than a month is a deleted test that still costs review time.
  • Covers a removed feature. Surprisingly common, and usually still passing because it asserts something trivially true.

Then actually delete them. This is the part teams cannot do, so say it plainly to your team: deleting a test is a legitimate engineering decision, not an admission of failure. A test that costs runtime on every commit and has never caught anything is a liability on the balance sheet. Keeping it out of sentiment is not rigour.

Disease three: the refactor nobody starts.

The suite needs restructuring. Everyone agrees. Nobody begins, because a big-bang refactor risks breaking everything and cannot be reviewed.

Refactor along a seam, incrementally. Pick one team's folder or one feature area. Migrate it fully to the new structure. Leave the rest alone. Both structures coexist for a while, which feels untidy and is far safer than the alternative.

The senior skill here is tolerating that temporary inconsistency and communicating it clearly, so nobody thinks the half-migrated state is the plan.

The measurement that makes all three arguments winnable. Track four numbers weekly and put them somewhere visible:

  • total runtime
  • flaky count (tests that passed only on retry)
  • failure rate per test
  • number of skipped tests

Without numbers, "the suite is bad" is a feeling and you will not get time to fix it. With numbers, you can show a trend line, and a trend line going the wrong way is an argument a manager can act on.

Remember this: entropy needs a process. Govern selectors with enforcement, delete tests without guilt, refactor along seams, and measure so you can argue.

Reference: Cypress best practices.