Scheduling: taints, tolerations, affinity

Scheduling: taints, tolerations, affinity

By default the scheduler puts a Pod wherever it fits. Three mechanisms let you change that, and they work in opposite directions.

The pair is asymmetric on purpose. A toleration permits but does not attract: a Pod that tolerates a taint may still land anywhere else. If you want it on those nodes, you need affinity as well.

Taints

kubectl taint node worker03 disk=ssd:NoSchedule       # add
kubectl taint node worker03 disk=ssd:NoSchedule-      # remove (trailing dash)
kubectl describe node control01 | grep -i taint

Three effects:

You already met a taint: kubectl cordon adds node.kubernetes.io/unschedulable:NoSchedule, and control plane nodes carry node-role.kubernetes.io/control-plane:NoSchedule. Taints are not exotic - they are the machinery under commands you have used all along.

Tolerations

tolerations:
  - key: disk
    operator: Equal
    value: ssd
    effect: NoSchedule

operator: Exists with no value tolerates any value of that key. Omit the key entirely with Exists and the Pod tolerates everything - that is how some monitoring DaemonSets run on every node no matter what.

With NoExecute you can add tolerationSeconds to say “evict me, but give me five minutes”.

nodeSelector

The simple version: exact label match, all of them.

kubectl label node worker01 disktype=ssd

spec:
  nodeSelector:
    disktype: ssd

Node affinity

The expressive version - operators (In, NotIn, Exists, Gt, Lt) and, crucially, a soft variant.

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:     # hard: must
      nodeSelectorTerms:
        - matchExpressions:
            - key: disktype
              operator: In
              values: ["ssd", "nvme"]
    preferredDuringSchedulingIgnoredDuringExecution:    # soft: try
      - weight: 100
        preference:
          matchExpressions:
            - key: topology.kubernetes.io/zone
              operator: In
              values: ["zone-a"]

Read the long names literally:

• requiredDuringScheduling - if nothing matches, the Pod stays Pending.
• preferredDuringScheduling - the scheduler scores nodes and prefers matches, but places the Pod anyway.
• IgnoredDuringExecution - once the Pod runs, changing node labels does not move it. Only a NoExecute taint evicts a running Pod.

Pod affinity and anti-affinity

Placement relative to other Pods rather than to nodes. Anti-affinity is the common one: keep my replicas off the same node.

affinity:
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchLabels: { app: web }
          topologyKey: kubernetes.io/hostname

topologyKey defines what “the same place” means - the same node, the same zone, the same rack.

Which one do I want?

References

• Taints and tolerations
• Assigning Pods to nodes
• Topology spread constraints