A test can pass and still be a problem. Let me show you the two ways that happens, because you will meet both.
Problem 1: the test races the app.
This is the fixed pause from the waiting lecture. A test with waitForTimeout(400) passes on your fast laptop and fails on the slower build server, maybe once in ten runs.
What makes this dangerous is not the failure. It is what the team learns. They see red, they re-run the build, it goes green. After a month, everyone re-runs red builds without looking at them.
I have seen how that story ends. A payment bug reached production on a Friday afternoon. The build had caught it. Three people re-ran that build without opening the report, because everyone had learned that red usually meant nothing.
The fix is the one you already know: wait for a condition, never for a number.
Problem 2: the test depends on another test.
This one is sneakier. Look:
let email; // shared between tests
test('signs in', async ({ page }) => {
email = 'demo@nimbus.app';
// ... uses email ...
});
test('rejects a wrong password', async ({ page }) => {
// ... also uses email ...
});
Run them in order and both pass. Run the second one alone and it fails, because email was never set.
The second test was never really passing. It was borrowing something the first test happened to leave behind.
This matters more than it looks, because Playwright runs test files at the same time by default, and build servers split suites across machines. As soon as the order changes, borrowed state disappears.
The fix is to give every test its own setup:
test('rejects a wrong password', async ({ page }) => {
const email = 'demo@nimbus.app'; // its own data
// ...
});
Here is a simple check you can run on any suite. Run one test on its own. It must pass. If it only passes when other tests run first, it is not finished.
Remember this: a test must pass alone, in any order, every time. Anything less and your team will stop believing the results.
Reference: Test isolation.