Lab 15 - Gateway API

Goal: install a Gateway controller and route two applications by host name through one port - then split traffic between two versions. Chapter: Gateway API

Gateway API resources are only data. Two things must exist first: the CRDs (they are not part of Kubernetes) and a controller that reads them.

Install the CRDs

# 1 - the standard channel: GatewayClass, Gateway, HTTPRoute
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.3.0/standard-install.yaml
kubectl get crd | grep gateway.networking
kubectl api-resources --api-group=gateway.networking.k8s.io

You have just extended the API server with new kinds - exactly the CRD mechanism covered on Day 3, met here in the wild. Check the releases page if a newer version is out.

# 2 - explain works on them immediately
kubectl explain gateway.spec.listeners
kubectl explain httproute.spec.rules.backendRefs

Install the controller

# 3 - NGINX Gateway Fabric. The data plane has no cloud LoadBalancer here, so
#     ask for NodePort and pin the port to 30080 for the port-80 listener.
helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric \
  --create-namespace -n nginx-gateway \
  --set nginx.service.type=NodePort \
  --set nginx.service.nodePorts[0].port=30080 \
  --set nginx.service.nodePorts[0].listenerPort=80
kubectl get pods -n nginx-gateway -w        # Ctrl-C when Running

If helm is missing:

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod +x get_helm.sh && sudo ./get_helm.sh && helm version

nodePorts is a list of {port, listenerPort} pairs: “expose node port 30080 for the Gateway listener on port 80”. A mapping whose listenerPort matches no listener is silently ignored - so if you later change the Gateway to listen on 8080, this entry stops applying and Kubernetes allocates a random port instead.

# 4 - what did that give you?
kubectl get gatewayclass
kubectl get all -n nginx-gateway

Two things to notice, because they cause the most confusion in this lab:

The GatewayClass named nginx is what your Gateway will point at.
The only Service here is ngf-nginx-gateway-fabric, and it is ClusterIP - correctly so. That is the control plane: the controller Pod, which talks to NGINX agents over gRPC. It is not the proxy and it never serves your traffic.

There is no proxy Pod and no NodePort Service yet at all.

Warning

The data plane does not exist until a Gateway exists, and it is not created in this namespace. NGINX Gateway Fabric 2.x provisions one NGINX Deployment and one Service per Gateway, in the Gateway’s own namespace. So after step 6 you will find them in default, not in nginx-gateway.

If you look for a NodePort Service in nginx-gateway now, you will find only the ClusterIP control plane service and conclude that --set nginx.service.type did nothing. It worked - you are looking in the wrong place at the wrong time.

Two applications

# 5 -
cd ~
kubectl create deploy shop --image nginx:1.27
kubectl create deploy wiki --image httpd:2.4
kubectl expose deploy shop --port 80
kubectl expose deploy wiki --port 80
kubectl get svc shop wiki

Both are ClusterIP. Nothing outside the cluster can reach them, and after this lab they still will not be reachable except through the Gateway.

The Gateway

# 6 -
cat > gateway.yaml <<'YAML'
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: web-gw
spec:
  gatewayClassName: nginx
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      allowedRoutes:
        namespaces:
          from: All
YAML
kubectl apply -f gateway.yaml
kubectl get gateway web-gw
kubectl describe gateway web-gw | tail -20

Watch PROGRAMMED. True means the controller accepted the Gateway and built a data plane for it; False means read the conditions in describe. Ingress had no equivalent - it just silently did nothing.

# 7 - NOW the data plane exists - in the Gateway's namespace
kubectl get pods,svc -n default -l gateway.networking.k8s.io/gateway-name=web-gw
kubectl get svc -A | grep -i web-gw

An NGINX Deployment and a Service named after the Gateway appeared in default, and the Service is NodePort with port 30080. Compare with nginx-gateway, which still holds only the ClusterIP control plane:

# 8 - the two halves, side by side
kubectl get svc -n nginx-gateway        # ClusterIP  - control plane
kubectl get svc -n default | grep web-gw   # NodePort - data plane, your traffic

# 9 - read the port from the cluster rather than trusting the pin
PORT=$(kubectl get svc -n default -l gateway.networking.k8s.io/gateway-name=web-gw \
  -o jsonpath='{.items[0].spec.ports[?(@.port==80)].nodePort}')
echo "gateway node port: $PORT"

Always read it back. If the pin did not apply - wrong listener port, an older chart, a typo in the --set - $PORT still gets you the real value, and every curl below keeps working.

Tip

If the Service is ClusterIP anyway, you can fix it without reinstalling:

SVC=$(kubectl get svc -n default -l gateway.networking.k8s.io/gateway-name=web-gw -o name)
kubectl patch $SVC -n default -p '{"spec":{"type":"NodePort"}}'
kubectl patch $SVC -n default --type=json \
  -p '[{"op":"replace","path":"/spec/ports/0/nodePort","value":30080}]'

That is a manual edit of an object the controller owns, so treat it as a workaround: the proper fix is helm upgrade with the right values, because the controller may reconcile your patch away.

The routes

# 10 - one route per application, owned by the application team
cat > routes.yaml <<'YAML'
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: shop
spec:
  parentRefs:
    - name: web-gw
  hostnames: ["shop.k8s.lab"]
  rules:
    - matches:
        - path: { type: PathPrefix, value: / }
      backendRefs:
        - name: shop
          port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: wiki
spec:
  parentRefs:
    - name: web-gw
  hostnames: ["wiki.k8s.lab"]
  rules:
    - matches:
        - path: { type: PathPrefix, value: / }
      backendRefs:
        - name: wiki
          port: 80
YAML
kubectl apply -f routes.yaml
kubectl get httproute
kubectl describe httproute shop | tail -15

Each route names its Gateway in parentRefs; the Gateway lists nobody. Adding an application means adding a route, never editing shared configuration.

# 11 - test: same port, different Host header
curl -s -H 'Host: shop.k8s.lab' http://worker01:$PORT | grep -o '<title>.*</title>'
curl -s -H 'Host: wiki.k8s.lab' http://worker01:$PORT | head -3
curl -s -o /dev/null -w '%{http_code}\n' -H 'Host: nothing.k8s.lab' http://worker01:$PORT

One port, two applications, routed by name. The 404 comes from the proxy itself - it got the request and had no matching route.

Traffic splitting

This is the part Ingress could not express without vendor annotations.

# 12 - a second version of the shop
kubectl create deploy shop-v2 --image httpd:2.4
kubectl expose deploy shop-v2 --port 80

kubectl patch httproute shop --type merge -p '{"spec":{"rules":[{"matches":[{"path":{"type":"PathPrefix","value":"/"}}],"backendRefs":[{"name":"shop","port":80,"weight":90},{"name":"shop-v2","port":80,"weight":10}]}]}}'

for i in $(seq 1 20); do
  curl -s -H 'Host: shop.k8s.lab' http://worker01:$PORT | grep -o -E 'nginx|It works'
done | sort | uniq -c

Roughly 18 to 2. Change the weights and re-run - that is a canary release in one field of one object.

Discovery

Infodiscovery

Task A. Route http://api.k8s.lab/wiki to the wiki Service while http://api.k8s.lab/ returns 404.

Show one answer

kubectl patch httproute wiki --type merge -p '{"spec":{"hostnames":["wiki.k8s.lab","api.k8s.lab"],"rules":[{"matches":[{"path":{"type":"PathPrefix","value":"/wiki"}}],"backendRefs":[{"name":"wiki","port":80}]}]}}'
curl -s -H 'Host: api.k8s.lab' http://worker01:$PORT/wiki | head -3
curl -s -o /dev/null -w '%{http_code}\n' -H 'Host: api.k8s.lab' http://worker01:$PORT/

The backend receives /wiki and httpd 404s on it - fix that with a URLRewrite filter, which is a typed field here rather than an annotation.

Task B. Send requests carrying the header x-version: beta to shop-v2 and everything else to shop. Header matching is something Ingress never had.

Show one answer

kubectl patch httproute shop --type merge -p '{"spec":{"rules":[{"matches":[{"headers":[{"name":"x-version","value":"beta"}]}],"backendRefs":[{"name":"shop-v2","port":80}]},{"matches":[{"path":{"type":"PathPrefix","value":"/"}}],"backendRefs":[{"name":"shop","port":80}]}]}}'
curl -s -H 'Host: shop.k8s.lab' -H 'x-version: beta' http://worker01:$PORT | grep -o 'It works'
curl -s -H 'Host: shop.k8s.lab' http://worker01:$PORT | grep -o '<title>.*</title>'

More specific rules win; order in the list breaks ties.

Task C. Make the hostnames work from support01 without a Host: header.

Show one answer

echo "$(getent hosts worker01 | awk '{print $1}') shop.k8s.lab wiki.k8s.lab api.k8s.lab" \
  | sudo tee -a /etc/hosts
curl -s http://shop.k8s.lab:$PORT | grep -o '<title>.*</title>'

In production this is DNS pointing at the load balancer in front of the nodes.

Task D. A route is not working. Using only kubectl, find out whether the Gateway accepted it, which Services it points at, and which namespace its data plane runs in.

Show one answer

kubectl describe httproute shop | tail -20        # Parents / conditions: Accepted, ResolvedRefs
kubectl get httproute shop -o jsonpath='{range .spec.rules[*].backendRefs[*]}{.name}{":"}{.port}{" "}{end}{"\n"}'
kubectl get gateway web-gw -o yaml | grep -A10 conditions
kubectl get pods,svc -A -l gateway.networking.k8s.io/gateway-name=web-gw
kubectl logs -n nginx-gateway -l app.kubernetes.io/name=nginx-gateway-fabric --tail=20

ResolvedRefs: False means the backend Service does not exist or the port is wrong - the status tells you, which is the practical advantage over Ingress. The last two commands separate the two halves: data plane in the Gateway’s namespace, controller logs in nginx-gateway.

Clean up

kubectl delete -f routes.yaml -f gateway.yaml
kubectl delete deploy shop wiki shop-v2
kubectl delete svc shop wiki shop-v2
sudo sed -i '/k8s.lab/d' /etc/hosts

# deleting the Gateway removed its data plane too - confirm
kubectl get pods,svc -n default | grep web-gw || echo "data plane gone"

Deleting the Gateway deleted the NGINX Deployment and Service with it: the controller owns them, exactly as a ReplicaSet owns its Pods.

Keep the controller and the CRDs installed - Lab 23 uses them.