The deploy job succeeds and exits zero, but production keeps serving the previous build. Where did the pipeline break?
The build stage builds and pushes the image; the deploy stage points the Deployment at it. Both jobs go green and exit zero, yet production still runs the old version, and re-running the deploy does nothing. Find the root cause in the config and fix it.
build:
stage: build
script:
- docker build -t registry.example.com/app:latest .
- docker push registry.example.com/app:latest
deploy-prod:
stage: deploy
script:
- kubectl set image deployment/app app=registry.example.com/app:latestExplain why the new build never goes live, and give the fix.
The mutable latest tag never changes the Deployment spec between releases, so kubectl set image sees no diff and triggers no rollout, nothing pulled. Fix it with immutable tags: push app:$CI_COMMIT_SHA and point the Deployment at it. Build once, promote that artifact — never rebuild per env.
- ✗Using a mutable
latesttag so the Deployment spec never changes - ✗Expecting a rollout when
kubectl set imagesees no diff - ✗Rebuilding a separate image per environment instead of promoting one
- →Why is build-once-promote-many safer than rebuilding for each environment?
- →How does
imagePullPolicyinteract with a reused mutable tag?
Solution
Cause
The latest tag is mutable. Both stages always use the identical string registry.example.com/app:latest. When kubectl set image sets the exact value already in the Deployment spec, the controller sees no diff — no new ReplicaSet, no rollout, no image pull. The job still exits zero because the command itself succeeded.
Fix — an immutable tag and build-once-promote-many
build:
stage: build
script:
- docker build -t registry.example.com/app:$CI_COMMIT_SHA .
- docker push registry.example.com/app:$CI_COMMIT_SHA
deploy-prod:
stage: deploy
script:
- kubectl set image deployment/app app=registry.example.com/app:$CI_COMMIT_SHA
- kubectl rollout status deployment/app # verify the rolloutA unique commit-SHA tag makes the spec string new on every release — the controller sees a diff, creates a new ReplicaSet, and pulls the new image. That same SHA-tagged image is then promoted into staging and prod: built once, the identical artifact ships. Never rebuild a separate image per environment, or prod runs a different binary than the one you tested.
