Lab 18 - Rolling updates and rollbacks

Lab 18 - Rolling updates and rollbacks

Goal: roll forward, break it, roll back - and choose a strategy. Chapter: Rolling updates and rollbacks

# 1 - a Deployment with 4 replicas
cd ~
kubectl create deploy web --image nginx:1.27 --replicas 4
kubectl annotate deploy web kubernetes.io/change-cause="initial 1.27"
kubectl rollout status deploy/web

# 2 - roll forward, watch it happen (second terminal: kubectl get pods -l app=web -w)
kubectl set image deploy/web nginx=nginx:1.28
kubectl annotate deploy web kubernetes.io/change-cause="upgrade to 1.28" --overwrite
kubectl rollout status deploy/web
kubectl get rs -l app=web
kubectl rollout history deploy/web

Two ReplicaSets: old at 0, new at 4. CHANGE-CAUSE makes the history readable.

# 3 - break it
kubectl set image deploy/web nginx=nginx:9.9-doesnotexist
kubectl rollout status deploy/web --timeout=30s     # does not finish
kubectl get pods -l app=web

Old Pods still Running - the rollout is stuck, the app is not down.

# 4 - make the failure visible faster
kubectl patch deploy web -p '{"spec":{"progressDeadlineSeconds":30}}'
sleep 40
kubectl get deploy web -o jsonpath='{range .status.conditions[*]}{.type}={.status} {.reason}{"\n"}{end}'

Progressing=False ProgressDeadlineExceeded - this is what a pipeline or monitoring should alert on.

# 5 - roll back
kubectl rollout undo deploy/web
kubectl rollout status deploy/web
kubectl rollout history deploy/web

The rollback became a new revision - revision numbers only go up.

Strategies

# 6 - zero downtime: never below 4, at most 1 extra
kubectl patch deploy web -p '{"spec":{"strategy":{"rollingUpdate":{"maxUnavailable":0,"maxSurge":1}}}}'
kubectl set image deploy/web nginx=nginx:1.27-alpine
kubectl get pods -l app=web -w          # Ctrl-C when done - READY never below 4

# 7 - Recreate: everything down, then up
kubectl patch deploy web --type=merge -p '{"spec":{"strategy":{"type":"Recreate","rollingUpdate":null}}}'
kubectl set image deploy/web nginx=nginx:1.28
kubectl get pods -l app=web -w          # all Terminating first, then new ones

Discovery

Infodiscover

Task A. Roll back to the revision that ran nginx:1.27 (not -alpine) explicitly by number.

kubectl rollout history deploy/web
kubectl rollout history deploy/web --revision=<n> | grep Image
kubectl rollout undo deploy/web --to-revision=<n>

Task B. Change the image and add an env variable COLOR=blue - but as one rollout, not two.

kubectl rollout pause deploy/web
kubectl set image deploy/web nginx=nginx:1.28-alpine
kubectl set env deploy/web COLOR=blue
kubectl rollout resume deploy/web
kubectl rollout history deploy/web        # only one new revision

Task C. Restart all Pods without changing anything. What changed in the Pod template?

kubectl rollout restart deploy/web
kubectl get deploy web -o jsonpath='{.spec.template.metadata.annotations}{"\n"}'

kubectl.kubernetes.io/restartedAt - a changed annotation is a changed template, so a rollout runs.

Clean up

kubectl delete deploy web