Lab 32 - Network policies with Calico

Lab 32 - Network policies with Calico

Goal: get to know the cluster’s CNI, then segment a three-tier app: only frontend -> backend -> db, nothing else. Chapter: Network policies and segmentation

Calico, up close

# 1 - the operator's view
kubectl get tigerastatus
kubectl get installation default -o yaml | grep -A8 calicoNetwork
kubectl get pods -n calico-system -o wide

# 2 - Calico resources, through the Calico API server
kubectl api-resources --api-group=projectcalico.org
kubectl get ippools -o wide

One IP pool 10.244.0.0/16, VXLAN encapsulation - the overlay you saw on Day 2.

A three-tier app

# 3 -
kubectl create ns shop
for app in frontend backend db; do
  kubectl run $app -n shop --image nginx:1.27 --labels app=$app --port 80
  kubectl expose pod $app -n shop --port 80
done
kubectl create ns other
kubectl run intruder -n other --image busybox:1.36 --command -- sleep 3600
kubectl run tester -n shop --image busybox:1.36 --labels app=tester --command -- sleep 3600
kubectl wait -n shop --for=condition=Ready pod --all --timeout=60s

# 4 - a small test helper: can <pod> in <ns> reach <target>?
cat > ~/reach.sh <<'SH'
#!/bin/sh
# usage: reach.sh <namespace> <pod> <url>
kubectl exec -n "$1" "$2" -- wget -qO- -T 2 "$3" >/dev/null 2>&1 && echo "OPEN    $2 -> $3" || echo "BLOCKED $2 -> $3"
SH
chmod +x ~/reach.sh
~/reach.sh shop frontend http://backend
~/reach.sh shop frontend http://db
~/reach.sh shop tester   http://db
~/reach.sh other intruder http://db.shop

Everything OPEN - including the intruder from another namespace straight to the database.

Default deny

# 5 -
cd ~
cat > deny.yaml <<'YAML'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: shop
spec:
  podSelector: {}
  policyTypes: ["Ingress"]
YAML
kubectl apply -f deny.yaml
~/reach.sh shop frontend http://backend        # BLOCKED
~/reach.sh other intruder http://db.shop       # BLOCKED

Allow exactly the flows you need

# 6 -
cat > allow.yaml <<'YAML'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backend-from-frontend
  namespace: shop
spec:
  podSelector:
    matchLabels: { app: backend }
  ingress:
    - from:
        - podSelector:
            matchLabels: { app: frontend }
      ports: [{ protocol: TCP, port: 80 }]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: db-from-backend
  namespace: shop
spec:
  podSelector:
    matchLabels: { app: db }
  ingress:
    - from:
        - podSelector:
            matchLabels: { app: backend }
      ports: [{ protocol: TCP, port: 80 }]
YAML
kubectl apply -f allow.yaml
~/reach.sh shop frontend http://backend        # OPEN
~/reach.sh shop backend  http://db             # OPEN
~/reach.sh shop frontend http://db             # BLOCKED
~/reach.sh shop tester   http://db             # BLOCKED
~/reach.sh other intruder http://db.shop       # BLOCKED

Egress - and the DNS trap

# 7 - frontend may only talk to backend
cat > egress.yaml <<'YAML'
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: frontend-egress
  namespace: shop
spec:
  podSelector:
    matchLabels: { app: frontend }
  policyTypes: ["Egress"]
  egress:
    - to:
        - podSelector:
            matchLabels: { app: backend }
      ports: [{ protocol: TCP, port: 80 }]
YAML
kubectl apply -f egress.yaml
~/reach.sh shop frontend http://backend        # BLOCKED?!

The name backend cannot be resolved - DNS (port 53 to CoreDNS) is egress too.

# 8 - allow DNS
cat >> egress.yaml <<'YAML'
    - to:
        - namespaceSelector:
            matchLabels: { kubernetes.io/metadata.name: kube-system }
          podSelector:
            matchLabels: { k8s-app: kube-dns }
      ports:
        - { protocol: UDP, port: 53 }
        - { protocol: TCP, port: 53 }
YAML
kubectl apply -f egress.yaml
~/reach.sh shop frontend http://backend        # OPEN
~/reach.sh shop frontend http://www.google.com # BLOCKED

Calico: a cluster-wide guard rail

Traefik’s internal port 8080 (API, dashboard, /ping) should never be reachable from workloads - whatever namespace policies teams write.

# 9 - before: any Pod reaches it
TRAEFIK=$(kubectl get pod -n traefik -l app.kubernetes.io/name=traefik -o jsonpath='{.items[0].status.podIP}')
~/reach.sh shop tester http://$TRAEFIK:8080/ping          # OPEN

# 10 - a platform rule, cluster-wide
cat > gnp.yaml <<'YAML'
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: protect-traefik-admin
spec:
  order: 10
  selector: app.kubernetes.io/name == 'traefik'
  types: ["Ingress"]
  ingress:
    - action: Deny
      protocol: TCP
      source:
        notNets: ["192.168.128.0/24"]   # nodes (kubelet probes) stay allowed
      destination:
        ports: [8080]
    - action: Allow                     # everything else, e.g. web traffic on 8000
YAML
kubectl apply -f gnp.yaml
~/reach.sh shop tester http://$TRAEFIK:8080/ping          # BLOCKED
curl -s -o /dev/null -w '%{http_code}\n' http://worker01:30080   # 404 - Traefik still serves
kubectl get pods -n traefik                               # still Ready

Deny and order are Calico features the standard API does not have. The final Allow matters: once a policy selects a Pod, anything no rule allows is dropped - without it Traefik would stop serving.

Discovery

Infodiscover

Task A. Open Whisker and find the denied flow intruder -> db.

kubectl port-forward -n calico-system service/whisker 8081:8081 --address 0.0.0.0 &
~/reach.sh other intruder http://db.shop

Browse to http://192.168.56.101:8081 from your PC, filter on namespace shop. Each flow shows source, destination, action and the policy that decided. kill %1 when done.

Task B. Allow Pods from namespace other - but only those labelled role=monitoring - to reach backend on port 80.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: { name: backend-from-monitoring, namespace: shop }
spec:
  podSelector: { matchLabels: { app: backend } }
  ingress:
    - from:
        - namespaceSelector:
            matchLabels: { kubernetes.io/metadata.name: other }
          podSelector:
            matchLabels: { role: monitoring }
      ports: [{ protocol: TCP, port: 80 }]

kubectl label pod intruder -n other role=monitoring
~/reach.sh other intruder http://backend.shop     # OPEN
~/reach.sh other intruder http://db.shop          # still BLOCKED

Namespace and Pod selector in one list item = both must match.

Task C. Write a Calico StagedNetworkPolicy that would deny all ingress to frontend and find out in Whisker what it would break - without breaking it.

apiVersion: projectcalico.org/v3
kind: StagedNetworkPolicy
metadata: { name: frontend-lockdown, namespace: shop }
spec:
  selector: app == 'frontend'
  types: ["Ingress"]

Traffic keeps flowing; Whisker shows the would-be verdicts as pending deny. Change kind to NetworkPolicy to enforce.

Clean up

kubectl delete ns shop other
kubectl delete -f gnp.yaml