Lab 19 - StatefulSets

Goal: stable names, per-Pod storage, ordered start - and the clean-up surprise. Chapter: StatefulSets

This lab needs the three PVs from Lab 18. Check they are Available:

kubectl get pv
kubectl patch pv pv-storage1 -p '{"spec":{"claimRef":null}}' 2>/dev/null
kubectl get pv

Headless Service first

# 1 -
cd ~
cat > web-headless.yaml <<'YAML'
apiVersion: v1
kind: Service
metadata:
  name: web-headless
spec:
  clusterIP: None            # headless - no virtual IP
  selector:
    app: web-sts
  ports:
    - port: 80
YAML
kubectl apply -f web-headless.yaml
kubectl get svc web-headless      # CLUSTER-IP is "None"

A normal Service hides the Pods behind one IP. A headless Service does the opposite: DNS returns the individual Pod addresses, so each replica is addressable by name. A StatefulSet requires one.

The StatefulSet

# 2 -
cat > web-sts.yaml <<'YAML'
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  serviceName: web-headless
  replicas: 3
  selector:
    matchLabels:
      app: web-sts
  template:
    metadata:
      labels:
        app: web-sts
    spec:
      containers:
        - name: web
          image: nginx:1.27
          ports:
            - containerPort: 80
          volumeMounts:
            - name: data
              mountPath: /usr/share/nginx/html
          resources:
            requests: { cpu: "20m", memory: "32Mi" }
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteMany"]
        storageClassName: ""
        resources:
          requests:
            storage: 500Mi
YAML
kubectl apply -f web-sts.yaml
kubectl get pods -l app=web-sts -w        # Ctrl-C once all three are Running

Watch the order: web-0 becomes Ready before web-1 is created, and web-1 before web-2. A Deployment starts all three at once; a StatefulSet is deliberately sequential, because a database replica cannot join a cluster that is not up yet.

# 3 - names and storage
kubectl get pods -l app=web-sts -o wide
kubectl get pvc
kubectl get pv

Note the PVC names: data-web-0, data-web-1, data-web-2 - the template name plus the Pod name. One PVC per replica, each bound to one of your three NFS PVs. This is why three replicas needed three PVs.

# 4 - stable identity
kubectl exec web-0 -- sh -c 'echo "I am web-0" > /usr/share/nginx/html/index.html'
kubectl exec web-1 -- sh -c 'echo "I am web-1" > /usr/share/nginx/html/index.html'
kubectl delete pod web-0
kubectl get pods -l app=web-sts           # web-0 comes back with the SAME name
kubectl exec web-0 -- cat /usr/share/nginx/html/index.html   # and the same data

A Deployment would have created web-7d9f...-xk2p with an empty volume. The StatefulSet recreated web-0 and reattached data-web-0.

# 5 - per-Pod DNS names
kubectl run tmp --image busybox --restart=Never -it --rm -- \
  sh -c 'nslookup web-0.web-headless.default.svc.cluster.local; wget -qO- web-1.web-headless'

<pod>.<service>.<namespace>.svc.cluster.local reaches one specific replica. That is how database members find each other.

# 6 - ordered, reverse scale-down
kubectl scale sts web --replicas=1
kubectl get pods -l app=web-sts           # web-2 goes first, then web-1
kubectl get pvc                           # the PVCs are all still here

Scaling down does not delete the storage. Scale back up and web-1 finds its old data.

# 7 -
kubectl scale sts web --replicas=2
kubectl exec web-1 -- cat /usr/share/nginx/html/index.html   # "I am web-1"

Clean up - the surprise

# 8 -
kubectl delete -f web-sts.yaml
kubectl get pvc                            # still there!
kubectl delete pvc data-web-0 data-web-1 data-web-2
kubectl delete -f web-headless.yaml
for p in pv-storage1 pv-storage2 pv-storage3; do
  kubectl patch pv $p -p '{"spec":{"claimRef":null}}'
done
kubectl get pv

Deleting a StatefulSet deletes the Pods and leaves the PVCs. That is deliberate - your data is the point of the object - and it means clean-up is always a manual second step.