Rolling updates and rollbacks

Rolling updates and rollbacks

Deployment lifecycle

stateDiagram-v2 [*] --> Progressing: new template (image, env, ...) Progressing --> Complete: all new Pods available Progressing --> Failed: progressDeadlineSeconds exceeded Complete --> Progressing: next change Failed --> Progressing: fix or rollout undo

Only a change of the Pod template starts a rollout. replicas does not.

kubectl rollout status deploy/web          # waits; exit code != 0 on failure
kubectl get deploy web -o jsonpath='{.status.conditions[*].type}'
ConditionMeaning
Progressing=Truerollout running or finished
Progressing=False reason ProgressDeadlineExceededstuck longer than progressDeadlineSeconds (600 s)
Available=Trueat least replicas - maxUnavailable Pods ready

Strategies

graph LR subgraph RollingUpdate A1[v1 v1 v1] --> A2[v1 v1 v2] --> A3[v1 v2 v2] --> A4[v2 v2 v2] end subgraph Recreate B1[v1 v1 v1] --> B2[- - -] --> B3[v2 v2 v2] end
spec:
  minReadySeconds: 5            # Pod must stay ready this long to count
  revisionHistoryLimit: 5       # old ReplicaSets kept for rollback
  strategy:
    type: RollingUpdate         # or Recreate (downtime, never two versions)
    rollingUpdate:
      maxSurge: 1               # extra Pods allowed during the rollout
      maxUnavailable: 0         # Pods allowed to be missing
SettingEffect
maxUnavailable: 0, maxSurge: 1zero downtime, needs room for one more Pod
maxUnavailable: 1, maxSurge: 0no extra capacity, one Pod less during rollout
Recreateall old Pods go first - for apps that cannot run two versions

A broken image stalls a rolling update - the old Pods keep serving.

History and rollback

kubectl annotate deploy/web kubernetes.io/change-cause="image 1.28"
kubectl rollout history deploy/web
kubectl rollout history deploy/web --revision=3
kubectl rollout undo deploy/web                    # previous revision
kubectl rollout undo deploy/web --to-revision=2
kubectl rollout pause deploy/web                   # batch several changes...
kubectl rollout resume deploy/web                  # ...into one rollout
kubectl rollout restart deploy/web                 # same spec, new Pods

undo copies the old template into the Deployment - it becomes a new revision. If the Deployment lives in git, fix git too.

Canary and blue/green with traffic weights: Gateway API (Day 2).

References