Shared and mutable test data
Tests depend on records created by other tests, so order and timing change outcomes.
Back to all QA problems
Problem breakdown
Why your Cypress or Playwright tests are flaky
Tests pass locally, fail in CI, and nobody trusts the suite. This page shows how to stop the randomness.
Symptoms
Signs your team is already paying the cost
A test fails in CI and passes on re-run without code changes.
Developers merge with red pipelines because failures are assumed to be noise.
The suite runtime increases because retries are covering instability instead of fixing it.
Nobody can explain if a failing test is product behavior, environment drift, or bad test design.
Root causes
Why this problem keeps coming back
UI selectors coupled to implementation details
Fragile selectors tied to CSS or position break whenever UI structure changes.
Environment drift between local and CI
Different runtime, timezone, seed data, or service mocks produce different behavior.
Retries used as strategy instead of guardrail
Retry settings mask instability, making the suite look green while quality degrades.
Remediation plan
Practical fixes your team can apply immediately
01
Make every test data-independent
Create test fixtures per test case and clean up deterministically instead of reusing shared accounts or records.
02
Adopt stable selectors and strict waiting
Prefer explicit test IDs and assertion-based waits over timing-based delays.
03
Align runtime and environment configuration
Pin versions, locale, timezone, and service mocks to remove cross-environment uncertainty.
04
Track and quarantine flakes with ownership
Tag flaky tests, assign owners, and enforce a policy that quarantined tests are fixed quickly.
Most teams do not need more tooling. They need clearer execution order, stable patterns, and ownership around reliability.
Diagnose flakiness by failure category
Start by classifying each flaky test: data contamination, selector instability, async timing, environment drift, or third-party dependency.
Without categories, teams fix symptoms. With categories, teams remove classes of failures and reduce maintenance cost permanently.
A 2-week stabilization plan
Week 1: isolate top failing tests, remove hard waits, and enforce data ownership per test. Week 2: lock runtime parity and establish flaky-test ownership metrics.
This creates a visible baseline where green checks are trustworthy and release decisions become faster.
Need help applying this in your pipeline?
If these issues are affecting delivery speed or release confidence, we can map a practical execution plan for your setup.
Frequently asked
Should I disable flaky tests until we fix them?+
Quarantine them with owner and deadline. Never silently disable them without visibility, or your coverage erodes without anyone noticing.
How many retries are acceptable?+
Use retries only as a temporary safety net. If retry dependence increases, treat it as a reliability incident and fix root causes.
Can we fix flaky tests without rewriting everything?+
Yes. Start with high-failure tests and high-value journeys. Most teams can get major stability gains with targeted refactoring and data isolation.
Related problems
How to fix a slow CI/CD pipeline
If feedback arrives after context is lost, quality drops. This page shows how to restore fast, trusted pipeline flow.
When no one wants to maintain your test framework
If one person owns all framework knowledge, delivery risk grows every sprint. This page shows how to reverse that.
Why your test environments are unreliable
If environment drift keeps invalidating results, your tests are not the main problem. This page explains the fix.