After a deploy, 287 of 942 e2e tests fail at once — diagnose from the CI output
Right after a routine production deploy, a large chunk of the e2e suite goes red. Nobody touched the tests in this release. You are handed only the CI run summary and one failure screenshot — you cannot rerun the suite yet.
Constraints: reason only from the evidence below; do not assume a specific product bug. Explain the single most-likely cause that produces this exact failure signature, and what you would check to confirm it.
CI run #4821 branch: main deploy: release-2.14.0
Suite: e2e-web 942 tests 287 FAILED 655 passed duration 41m
FAILED (sample):
checkout/adds_item_to_cart TimeoutException: waited 30s for [data-test=cart-badge]
checkout/applies_promo TimeoutException: waited 30s for [data-test=promo-input]
account/updates_profile TimeoutException: waited 30s for [data-test=save-btn]
... 284 more, all TimeoutException waiting for element presence
PASSED (sample): login/*, static-pages/*, health/ping
Screenshot (checkout/adds_item_to_cart): page body reads
"502 Bad Gateway — upstream connect error"
First failure: 14:02:11 · Deploy finished: 14:02:04Diagnose the cause.
These are not 287 independent regressions — they share one signature: every failure is a timeout waiting for an element, the screenshot shows a 502 Bad Gateway, and the first failure is seconds after the deploy finished. The app was not ready when the suite started. It is an environment-readiness problem: add a health-check gate before e2e, then rerun.
- ✗Reading mass failures as many independent regressions
- ✗Ignoring that every failure shares one error signature
- ✗Missing the 502 and the deploy-to-failure timing
- →What health signal should gate the suite start after a deploy?
- →How would you confirm the app, not the tests, was at fault?
The diagnosis comes from the signature, not the individual tests.
Group the failures — they are all the same:
287 × TimeoutException: waited 30s for [data-test=...]
screenshot → "502 Bad Gateway — upstream connect error"
first failure 14:02:11 · deploy finished 14:02:04 (Δ = 7s)
passed: login/*, static-pages/*, health/pingOne error class across all of them, a 502 from the upstream, and a start 7 seconds after the deploy mean the app (or a dependency) was still coming up when the run began. Only static pages, login, and health passed — they never hit the failing upstream.
These are not 287 regressions. Confirm it: a rerun after warm-up is usually green. Fix it in the pipeline — wait for readiness before e2e:
steps:
- deploy: release-2.14.0
- wait_for:
url: https://app/health
status: 200
timeout: 300s
- run: e2e-web
