Lab 29 - Workload hardening

Goal: take a default container apart, then run the same web server hardened: non-root, read-only, no capabilities. Chapter: Security context and container hardening

What a default container can do

# 1 -
kubectl create ns hard
kubectl run plain -n hard --image nginx:1.27
kubectl exec -n hard plain -- id                               # uid=0(root)
kubectl exec -n hard plain -- sh -c 'echo hacked > /usr/share/nginx/html/index.html; echo written'
kubectl exec -n hard plain -- grep Cap /proc/1/status

Root, writable filesystem, a set of capabilities (CapEff not zero). If an attacker gets code execution, they can rewrite your website.

Just switch it on?

# 2 - nginx with runAsNonRoot
kubectl run strict -n hard --image nginx:1.27 \
  --overrides='{"spec":{"securityContext":{"runAsNonRoot":true}}}'
kubectl get pod strict -n hard
kubectl describe pod strict -n hard | grep -i -A2 error

CreateContainerConfigError: container has runAsNonRoot and image will run as root. The image itself must be built for non-root.

The hardened version

# 3 -
cd ~
cat > hardened.yaml <<'YAML'
apiVersion: v1
kind: Pod
metadata:
  name: hardened
  namespace: hard
spec:
  automountServiceAccountToken: false
  securityContext:
    runAsNonRoot: true
    runAsUser: 101
    seccompProfile: { type: RuntimeDefault }
  containers:
    - name: web
      image: nginxinc/nginx-unprivileged:1.27
      ports: [{ containerPort: 8080 }]
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        capabilities: { drop: ["ALL"] }
      resources:
        requests: { cpu: "50m", memory: "32Mi" }
        limits:   { memory: "64Mi" }
      volumeMounts:
        - { name: tmp, mountPath: /tmp }
  volumes:
    - { name: tmp, emptyDir: {} }
YAML
kubectl apply -f hardened.yaml
kubectl get pod hardened -n hard

# 4 - verify
kubectl exec -n hard hardened -- id                               # uid=101
kubectl exec -n hard hardened -- sh -c 'echo hacked > /usr/share/nginx/html/index.html'   # Read-only file system
kubectl exec -n hard hardened -- grep CapEff /proc/1/status       # 0000000000000000
kubectl exec -n hard hardened -- sh -c 'echo scratch > /tmp/x && cat /tmp/x'  # /tmp is writable
kubectl port-forward -n hard pod/hardened 8080:8080 &
curl -s localhost:8080 | grep -o '<title>.*</title>'; kill %1

Same web server, same page - and an attacker inside finds: no root, no write access, no capabilities, no API token.

Discovery

Infodiscover

Task A. Without readOnlyRootFilesystem exceptions nginx needs /tmp. Remove the tmp volume and apply again. What breaks, and where do you see it?

Show one answer

kubectl delete -f hardened.yaml
sed -i '/volumeMounts:/,$d' hardened.yaml
kubectl apply -f hardened.yaml
kubectl logs -n hard hardened        # mkdir() "/tmp/client_temp" failed (30: Read-only file system)

Read-only root plus explicit writable emptyDir mounts is the pattern. Restore the file afterwards (re-run step 3).

Task B. Why does ping fail in a container with capabilities.drop: [ALL]?

Show one answer

kubectl run pinger -n hard --image busybox:1.36 --restart=Never \
  --overrides='{"spec":{"containers":[{"name":"pinger","image":"busybox:1.36","command":["ping","-c1","1.1.1.1"],"securityContext":{"capabilities":{"drop":["ALL"]}}}]}}'
kubectl logs -n hard pinger          # permission denied

ICMP needs raw sockets (NET_RAW). Dropping it is also what stops ARP/ICMP tricks from inside a compromised Pod.

Clean up

Keep namespace hard and hardened.yaml - Lab 30 uses them.

kubectl delete pod plain strict pinger -n hard --ignore-not-found