Troubleshooting routine

Troubleshooting routine

One order of operations, every time. It is short because it works.

graph TB A["kubectl get pod -o wide<br/>what state? which node?"] --> B["kubectl describe pod<br/>read the Events at the bottom"] B --> C{"did the container start?"} C -->|no| D["fix from the event:<br/>image, scheduling, volume, probe"] C -->|yes| E["kubectl logs [--previous]<br/>what did the app say?"] E --> F["kubectl exec -it -- sh<br/>only if it is running"]

The states and what they mean

StateWhere the answer is
Pendingdescribe -> FailedScheduling: no capacity, a taint, an unbound PVC
ContainerCreatingdescribe -> image pull or volume mount in progress, or stuck
ImagePullBackOff / ErrImagePullwrong image name or tag, or registry credentials
CrashLoopBackOfflogs --previous - the app started and exited
Running but not Readythe readiness probe is failing; the Service will not send traffic
OOMKilleddescribe -> Last State; memory limit too low
Terminating forevera finalizer, or a container ignoring SIGTERM

Wider than one Pod

kubectl get events --sort-by=.lastTimestamp | tail -20
kubectl get events -A --field-selector type=Warning
kubectl get pods -A -o wide | grep -v Running
kubectl top nodes ; kubectl top pods

When the application answers nothing

Work outwards from the Pod: is the Pod ready, does the Service have endpoints, does DNS resolve, does the Gateway route.

kubectl get endpointslices -l kubernetes.io/service-name=web-svc

Empty endpoints is the single most common cause, and it is always a label mismatch or a failing readiness probe - not the network.

References