Lab Extra - 01 - External access from your PC

Lab Extra - 01 - External access from your PC

Goal: open http://red.app.internal, http://blue.app.internal and http://green.app.internal in the browser on your Windows PC and land on three different applications in the cluster - through an edge proxy on support01, a MetalLB load-balancer address and the Gateway API. Chapters: Services · Helm · Gateway API

Info

Homework. This lab does not fit into the course day - plan 2-3 hours. It needs Traefik, the Gateway API CRDs and the Gateway traefik-gateway from Lab 14 and Lab 16, and nothing from later days. Later labs do not depend on it: NodePort 30080 keeps working whatever you do here.

The big picture

So far you tested everything from support01 with curl -H 'Host: ...' http://worker01:30080. Real users do not log in to a jump host. They type a name into a browser, somewhere outside the cluster network. In this lab your Windows PC plays that user.

 YOUR PC (Windows)             support01 = the EDGE            KUBERNETES CLUSTER
 192.168.56.1                  two network cards               192.168.128.0/24 only
+----------------------+      +--------------------------+    +-----------------------------------+
| browser              |      | enp0s8  192.168.56.101   |    |  MetalLB address 192.168.128.150  |
| red.app.internal     | ---> |   HAProxy :80            | -> |    = Service traefik (LoadBalancer)|
|                      |      |   (reverse proxy)        |    |  Traefik  -> Gateway traefik-gateway|
| hosts file:          |      | enp0s9  192.168.128.100  |    |    HTTPRoute red   -> Service red  |
| *.app.internal ->    |      +--------------------------+    |    HTTPRoute blue  -> Service blue |
|   192.168.56.101     |                                      |    HTTPRoute green -> Service green|
+----------------------+                                      +-----------------------------------+
   network 192.168.56.0/24 (host-only)          network 192.168.128.0/24 (private, cluster only)

Why this shape? Your PC can reach only 192.168.56.101. The cluster network 192.168.128.0/24 is private - exactly like a datacenter where users reach a DMZ, and only the DMZ reaches the servers behind it. support01 has a leg in both networks, so it becomes the edge.

Each hop has a different owner - this is how it looks in a real company:

HopComponentOwned byKnows about
name -> IPhosts file (in real life: DNS)network / DNS team*.app.internal points to the edge
edgeHAProxy on support01network teamone address in the cluster, nothing about apps
cluster entryMetalLB + Service traefikplatform teamone stable IP for the Gateway
routingGateway + HTTPRoutesplatform + app teamswhich host name goes to which Service
appDeployments red, blue, greenapp teamtheir own Pods

Adding a fourth application later must only touch the last two rows. Keep that in mind - it is the test of a good design.

You will build it in two variants, like many real platforms evolve:

Variant A (Part 3):  HAProxy -> worker01:30080, worker02:30080, worker03:30080   (NodePorts)
Variant B (Part 6):  HAProxy -> 192.168.128.150:80                               (MetalLB)
PartWhat you doWhere
0check the starting pointsupport01
1three apps with HTTPRoutessupport01 (kubectl)
2host names on WindowsPC
3HAProxy -> NodePorts (variant A)support01
4MetalLB, the LoadBalancer for bare metalsupport01 (kubectl, helm)
5Traefik becomes a LoadBalancer Servicesupport01 (helm)
6HAProxy -> MetalLB address (variant B)support01
Optional Afailover of the MetalLB addressVirtualBox + support01
Optional BTLS at the edgesupport01 + PC
Optional Cwho is the client? X-Forwarded-Forsupport01

Part 0 - Where we start

# 1 - Traefik, the Gateway and the NodePort from Lab 14/16
kubectl get svc -n traefik
kubectl get gateway -n traefik
curl -s -o /dev/null -w '%{http_code}\n' http://worker01:30080/

Expected: Service traefik of type NodePort with 80:30080, Gateway traefik-gateway with PROGRAMMED True, and HTTP 404 (Traefik answers, no route matches). If any of it is missing, redo Lab 14 steps 4-7.

# 2 - the node addresses - you will need them
kubectl get nodes -o wide
ip -brief address show enp0s9          # support01's own leg in the cluster network

Note the INTERNAL-IP of every node. MetalLB will use 192.168.128.150-200, which must not collide with any of them.

Part 1 - Three applications

Concept: one IP, many sites - the Host header

How can three sites share one IP address and one port? The browser tells the server which site it wants. Every HTTP/1.1 request carries a Host header:

GET / HTTP/1.1
Host: red.app.internal          <- the name you typed, not the IP
User-Agent: Mozilla/5.0 ...
Accept: text/html

The TCP connection goes to an IP address, but the proxies along the way read Host and choose the backend. That is what an HTTPRoute’s hostnames field matches on - you did it in Lab 16 with curl -H 'Host: shop.k8s.lab'.

                                 +-> Host: red.app.internal   -> Service red
request to 192.168.128.150:80 ---+-> Host: blue.app.internal  -> Service blue
                                 +-> Host: green.app.internal -> Service green

The apps

We use podinfo, a small demo web app. Its colour and message come from environment variables, so one image gives three visibly different apps, and its /headers endpoint shows what the app actually received - handy for looking at the traffic from the inside.

# 3 - namespace, three Deployments and three Services
cd ~
kubectl create namespace colors

for c in red blue green; do
  case $c in
    red)   hex='#c0392b' ;;
    blue)  hex='#2e6fd8' ;;
    green) hex='#2e8b57' ;;
  esac
  cat <<YAML
apiVersion: apps/v1
kind: Deployment
metadata:
  name: $c
  namespace: colors
spec:
  replicas: 2
  selector:
    matchLabels: { app: $c }
  template:
    metadata:
      labels: { app: $c }
    spec:
      containers:
        - name: podinfo
          image: ghcr.io/stefanprodan/podinfo:6.7.0
          ports:
            - containerPort: 9898
          env:
            - { name: PODINFO_UI_COLOR,   value: "$hex" }
            - { name: PODINFO_UI_MESSAGE, value: "Hello from $c" }
---
apiVersion: v1
kind: Service
metadata:
  name: $c
  namespace: colors
spec:
  selector: { app: $c }
  ports:
    - port: 80
      targetPort: 9898
---
YAML
done > colors.yaml

kubectl apply -f colors.yaml
kubectl get deploy,svc -n colors

Nothing new: a Deployment and a ClusterIP Service per colour, the Service port 80 forwarding to the container port 9898.

# 4 - one HTTPRoute per app - the app team's side of the Gateway
for c in red blue green; do
  cat <<YAML
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: $c
  namespace: colors
spec:
  parentRefs:
    - name: traefik-gateway
      namespace: traefik
  hostnames: ["$c.app.internal"]
  rules:
    - backendRefs:
        - name: $c
          port: 80
---
YAML
done > routes.yaml

kubectl apply -f routes.yaml
kubectl get httproute -n colors

# 5 - test from support01, the way you know from Lab 16
for c in red blue green; do
  curl -s -H "Host: $c.app.internal" http://worker01:30080/ | grep '"message"'
done

Three different messages: routing by host name works inside the cluster. Everything from here on is about getting a browser outside to this point.

Part 2 - Host names on your Windows PC

Concept: from name to address

Before the browser can open a connection it needs an IP address for red.app.internal. Windows asks, in this order:

browser: "red.app.internal?"
   |
   v
1. hosts file   C:\Windows\System32\drivers\etc\hosts   -> found? done.
   |
   v
2. DNS server   (from your network settings)            -> real life ends here

In a company, the DNS team would create one wildcard record *.app.internal -> edge address and every new app would work at once. We do not run a DNS server for the lab, so we use the hosts file. It has no wildcards - every name needs its own entry.

Why .internal? In 2024 ICANN reserved .internal for private networks; it will never be sold on the internet, so your names can never clash with a real site. (.local is taken by multicast DNS and causes odd delays; do not invent TLDs like .lab either - they may become real.)

Why 192.168.56.101? That is the only address your PC can reach. The names point to the edge, not to the cluster.

Open PowerShell as Administrator on your PC (Start -> type powershell -> Run as administrator):

# 6 - add the names (PowerShell as Administrator, on your PC)
Add-Content -Path "$env:windir\System32\drivers\etc\hosts" `
  -Value "192.168.56.101`tred.app.internal blue.app.internal green.app.internal"
Get-Content "$env:windir\System32\drivers\etc\hosts" | Select-String "app.internal"
ipconfig /flushdns

# 7 - does the name resolve, and is something listening?
ping -n 1 red.app.internal
Test-NetConnection red.app.internal -Port 80

ping shows 192.168.56.101 - the name works. Test-NetConnection shows TcpTestSucceeded : False - nothing listens on port 80 of support01 yet. That is Part 3.

Tip

If your browser is configured with a proxy server (common on company laptops), add *.app.internal to its bypass list, otherwise the request goes to that proxy and never reaches support01.

Part 3 - The edge proxy, variant A: HAProxy -> NodePorts

Concept: reverse proxy at the edge

A forward proxy acts for clients (“let me go out to the internet for you”). A reverse proxy acts for servers: clients talk to it as if it were the site, and it passes the request to the real servers behind it.

 client ------> [ reverse proxy ] ------> server 1
                  one public name  \----> server 2
                  health checks     \---> server 3
                  TLS, logging, limits

Every real platform has one at the edge - a hardware load balancer, HAProxy, nginx, a cloud load balancer. We use HAProxy, the most common open-source one.

HAProxy’s configuration has two halves:

frontend  = where HAProxy listens        (bind 192.168.56.101:80)
backend   = where it sends the traffic   (a list of servers + health checks)

In mode http HAProxy understands HTTP (it can read and add headers, answer with errors, redirect). In mode tcp it only moves bytes. We start with http.

Install and configure

# 8 - install HAProxy on support01 and keep the packaged config
sudo apt-get update
sudo apt-get install -y haproxy
haproxy -v
sudo cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.orig

# 9 - variant A: the edge knows every worker node
sudo tee /etc/haproxy/haproxy.cfg >/dev/null <<'CFG'
global
    log /dev/log local0
    user haproxy
    group haproxy

defaults
    mode http
    log global
    option httplog
    option forwardfor              # add X-Forwarded-For: <client IP>
    timeout connect 5s
    timeout client  30s
    timeout server  30s

frontend apps_http
    bind 192.168.56.101:80         # only the outside leg
    default_backend k8s_nodeports

backend k8s_nodeports
    balance roundrobin
    server worker01 worker01:30080 check
    server worker02 worker02:30080 check
    server worker03 worker03:30080 check

listen stats
    bind 192.168.56.101:8404
    stats enable
    stats uri /
    stats refresh 5s
CFG

sudo haproxy -c -f /etc/haproxy/haproxy.cfg     # always validate first
sudo systemctl restart haproxy
sudo systemctl status haproxy --no-pager | head -5

check makes HAProxy open a TCP connection to each server every 2 seconds; a server that does not answer is taken out of rotation. If worker0x did not resolve in Lab 01, write its IP address instead.

# 10 - test the edge from support01 (same path as your PC will use)
curl -s -H 'Host: red.app.internal' http://192.168.56.101/ | grep '"message"'

Now on your PC, open http://red.app.internal, http://blue.app.internal and http://green.app.internal in the browser. Three colours, three messages. Reload a few times and watch the hostname field change - two Pods behind each Service.

Open http://192.168.56.101:8404 - the HAProxy stats page. Three green servers, requests spread over them.

# 11 - the access log: one line per request
sudo tail -n 5 /var/log/haproxy.log

What really happens - the double hop

PC --> HAProxy --> worker01:30080 --(kube-proxy)--> Traefik Pod on worker03
                                                         |
                                           Traefik --> red Pod on worker02

A NodePort is open on every node, whether a Traefik Pod runs there or not. kube-proxy forwards the packet to a Traefik Pod, possibly on another node. So HAProxy can pick any worker, and one extra hop inside the cluster is the price.

Why this is not the end state

Look at the backend: the edge knows the list of worker nodes. Add a worker, remove one for maintenance, change an IP - and somebody from the network team has to edit the edge. In most companies that is a change request, a ticket and a wait. The cluster should hand the edge one stable address instead. That is what a Service of type LoadBalancer is for.

Part 4 - MetalLB: a LoadBalancer for bare metal

Concept: the Service types stack

LoadBalancer   external IP 192.168.128.150:80   <- who provides this IP?
    |  includes
NodePort       <every node>:30080
    |  includes
ClusterIP      10.96.x.x:80                     <- kube-proxy on every node
    |
Pods

Each type builds on the one below. A LoadBalancer Service still has a NodePort and a ClusterIP. Kubernetes itself does not provide the external IP - it only asks for one. In a cloud, the cloud controller creates a cloud load balancer. On bare metal, nobody answers, and the Service waits forever.

# 12 - see the problem first
kubectl expose deployment red -n colors --name red-lb \
  --type LoadBalancer --port 80 --target-port 9898
kubectl get svc red-lb -n colors

EXTERNAL-IP <pending> - and it would stay so. Leave it; MetalLB will fix it.

Concept: how MetalLB works in L2 mode

MetalLB has two parts:

controller  (Deployment, 1 Pod)     "IPAM": takes a free address from YOUR pool
                                    and writes it into the Service status
speaker     (DaemonSet, 1 per node) announces the address on the network

The speaker is a DaemonSet - the pattern from Lab 09: an agent on every node.

How does a packet for 192.168.128.150 find a node? Machines in one Ethernet network do not send to IP addresses, they send to MAC addresses. To learn the MAC for an IP they ask the whole network with ARP:

support01:  "Who has 192.168.128.150? Tell 192.168.128.100"   (broadcast to all)
            |
            +--> control01: not mine, silence
            +--> worker01:  not mine, silence
            +--> worker02:  speaker answers "192.168.128.150 is at 08:00:27:aa:bb:cc"
            +--> worker03:  not mine, silence

support01 caches:  192.168.128.150 -> 08:00:27:aa:bb:cc  (= worker02)
all traffic for .150 now goes to worker02; kube-proxy there treats it like
the Service IP and forwards to a Pod - on worker02 or any other node.

The address is not configured on any network card - a speaker simply answers ARP for it. Three consequences you must know:

FactMeaning
one node at a timeL2 mode gives failover, not load balancing: all traffic for one address enters through one node
failoverif that node dies, another speaker takes over and sends a gratuitous ARP (“the address is now at my MAC”); a few seconds of errors
same network onlyARP does not cross routers - the pool must be in the nodes’ own subnet. For routed networks MetalLB has a BGP mode

Where do the addresses come from? From you. MetalLB does not invent addresses; you give it a range that nobody else uses. In a company, the network team reserves it. Here: 192.168.128.150 - 192.168.128.200.

Install

# 13 - which mode does kube-proxy run in?
kubectl get configmap kube-proxy -n kube-system -o jsonpath='{.data.config\.conf}' \
  | grep -E '^mode|strictARP'

mode: "" (empty = iptables) - nothing to do. With mode: ipvs you would have to set strictARP: true first, otherwise every node answers ARP for the addresses and traffic goes to random nodes.

# 14 - MetalLB with Helm (pin the version your trainer gives you)
helm repo add metallb https://metallb.github.io/metallb
helm repo update
helm install metallb metallb/metallb -n metallb-system --create-namespace \
  --version 0.16.1 --wait
kubectl get deploy,ds,pods -n metallb-system -o wide

One controller Pod, and speaker Pods on the nodes. The control plane node carries the label node.kubernetes.io/exclude-from-external-load-balancers (kubeadm sets it), so control01 never announces addresses.

# 15 - MetalLB's own resources are CRDs, like Gateway API
kubectl api-resources --api-group=metallb.io

Give MetalLB its addresses

# 16 - the pool and the announcement
cat > metallb-pool.yaml <<'YAML'
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: lab-pool
  namespace: metallb-system
spec:
  addresses:
    - 192.168.128.150-192.168.128.200
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: lab-l2
  namespace: metallb-system
spec:
  ipAddressPools: ["lab-pool"]
YAML
kubectl apply -f metallb-pool.yaml
kubectl get ipaddresspool,l2advertisement -n metallb-system

Two objects on purpose: the pool says which addresses, the advertisement says how to announce them (L2 here, BGP elsewhere).

# 17 - the pending Service from step 12
kubectl get svc red-lb -n colors

EXTERNAL-IP 192.168.128.150 (the first free one) - without you touching the Service. The controller watched for Services of type LoadBalancer, found one without an address and assigned it. A control loop, like everything else.

# 18 - which node announces it?
kubectl describe svc red-lb -n colors | grep -A5 Events

The event nodeAssigned names the node, e.g. announcing from node "worker02" with protocol "layer2".

# 19 - ask it directly, then look at support01's ARP cache
curl -s http://192.168.128.150/ | grep '"message"'
ip neigh show 192.168.128.150

The ARP entry holds a MAC address. Prove it belongs to the announcing node:

# 20 - whose MAC is it? (password: linux)
MAC=$(ip neigh show 192.168.128.150 | awk '{print $5}')
for n in worker01 worker02 worker03; do
  echo "$n: $(ssh student@$n ip -brief link | grep -i "$MAC")"
done

Only the node from the event shows the MAC. No network card in the cluster has 192.168.128.150 configured - check with ssh student@worker02 ip -brief address.

red-lb was a demonstration: an address per Service works, and is the right choice for non-HTTP services (a database, MQTT, SSH). For HTTP we want one address for the Gateway and routing by host name behind it. Give the address back:

# 21 -
kubectl delete svc red-lb -n colors

Part 5 - Traefik becomes a LoadBalancer Service

The Traefik Service is the door of the Gateway. We change its type to LoadBalancer and pin its address: the edge configuration will point to it, so it must never change - not on reinstall, not on upgrade.

# 22 - the values from Lab 14, plus three changes
cat > traefik-values-lb.yaml <<'YAML'
deployment:
  replicas: 2                               # change 1: survive a node failure
service:
  type: LoadBalancer                        # change 2: ask for an external IP
  annotations:
    metallb.io/loadBalancerIPs: 192.168.128.150   # change 3: pin it
ports:
  web:
    nodePort: 30080                         # kept: Labs 15-16 and later still work
  websecure:
    nodePort: 30443
providers:
  kubernetesGateway:
    enabled: true
gateway:
  listeners:
    web:
      namespacePolicy:
        from: All
YAML

# 23 - upgrade with the SAME chart version, only the values change
CHART_VERSION=$(helm get metadata traefik -n traefik | awk '/^VERSION:/{print $2}')
echo "$CHART_VERSION"
helm upgrade traefik traefik/traefik -n traefik --version "$CHART_VERSION" \
  -f traefik-values-lb.yaml --wait
helm history traefik -n traefik | tail -2
kubectl get svc traefik -n traefik

Why --version? Without it helm upgrade takes the newest chart in the repository - you wanted to change a Service type, not upgrade Traefik on the side. One change at a time.

The Service now shows TYPE LoadBalancer, EXTERNAL-IP 192.168.128.150 and still 80:30080 - the stack from Part 4 in one line.

# 24 - both doors work
curl -s -H 'Host: blue.app.internal' http://192.168.128.150/ | grep '"message"'
curl -s -H 'Host: blue.app.internal' http://worker01:30080/  | grep '"message"'
Note

Older MetalLB versions use the annotation metallb.universe.tf/loadBalancerIPs. If the Service gets a random address from the pool instead of .150, check kubectl describe svc traefik -n traefik for MetalLB events and the annotation name for your version.

Part 6 - The edge proxy, variant B: HAProxy -> MetalLB

# 25 - the edge now knows one address, not the nodes
sudo tee /etc/haproxy/haproxy.cfg >/dev/null <<'CFG'
global
    log /dev/log local0
    user haproxy
    group haproxy

defaults
    mode http
    log global
    option httplog
    option forwardfor
    timeout connect 5s
    timeout client  30s
    timeout server  30s

frontend apps_http
    bind 192.168.56.101:80
    # the edge only serves the domain it is responsible for
    http-request deny deny_status 404 unless { hdr_end(host) -i .app.internal }
    default_backend k8s_gateway

backend k8s_gateway
    server gateway 192.168.128.150:80 check

listen stats
    bind 192.168.56.101:8404
    stats enable
    stats uri /
    stats refresh 5s
CFG

sudo haproxy -c -f /etc/haproxy/haproxy.cfg
sudo systemctl reload haproxy

reload, not restart: HAProxy starts new workers with the new config and lets the old ones finish their connections - no request is dropped. That is how you change an edge in production.

# 26 - allowed and refused host names
curl -s -o /dev/null -w '%{http_code}\n' -H 'Host: green.app.internal' http://192.168.56.101/
curl -s -o /dev/null -w '%{http_code}\n' -H 'Host: evil.example.com'   http://192.168.56.101/

200 and 404. The second request never reached the cluster.

On your PC, reload the three sites - they work exactly as before. The difference is invisible to users and big for operations:

Variant A                                   Variant B
HAProxy backend: worker01, 02, 03           HAProxy backend: 192.168.128.150
new worker node -> edge change              new worker node -> nothing to do
extra hop via any node                      one entry node (the announcer)
no extra component                          MetalLB in the cluster

Prove the design: add a fourth app

The test from the big picture: a new application must not touch the edge.

# 27 - app team only: Deployment, Service, HTTPRoute
kubectl create deployment yellow -n colors --image ghcr.io/stefanprodan/podinfo:6.7.0
kubectl set env deployment/yellow -n colors PODINFO_UI_COLOR='#d4ac0d' PODINFO_UI_MESSAGE='Hello from yellow'
kubectl expose deployment yellow -n colors --port 80 --target-port 9898
cat <<'YAML' | kubectl apply -f -
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata: { name: yellow, namespace: colors }
spec:
  parentRefs: [{ name: traefik-gateway, namespace: traefik }]
  hostnames: ["yellow.app.internal"]
  rules: [{ backendRefs: [{ name: yellow, port: 80 }] }]
YAML
curl -s -H 'Host: yellow.app.internal' http://192.168.56.101/ | grep '"message"'

It works through the edge immediately. For the browser, add yellow.app.internal to the hosts file on your PC - with a wildcard DNS record even that step would disappear.

Optional A - Failover of the MetalLB address

What happens when the node that announces .150 dies?

# 28 - which node is it now?
kubectl describe svc traefik -n traefik | grep nodeAssigned
kubectl get pods -n traefik -o wide

In a second terminal on support01, start a request loop through the whole chain:

# 29 - one character per request: 200 = ok, 000 = no answer
while true; do
  curl -s -m 1 -o /dev/null -w '%{http_code} ' -H 'Host: red.app.internal' http://192.168.56.101/
  sleep 0.5
done

Now in VirtualBox, power off that worker (right-click -> Close -> Power Off). Watch the loop, then:

# 30 - a new announcer, a new MAC
kubectl describe svc traefik -n traefik | grep nodeAssigned
ip neigh show 192.168.128.150

Typically you see a few seconds of 000 or 502, then 200 again. Two different things recover at different speeds:

MetalLB address moves          seconds    (speakers notice the node is gone,
                                           the new one sends a gratuitous ARP)
dead Pods leave the Services   ~40 s      (the node becomes NotReady, its
                                           Pods' endpoints are removed)

That is why step 22 set Traefik to 2 replicas: with one replica on the dead node, the Gateway would be down until Kubernetes starts a new Pod elsewhere. Power the worker on again afterwards and stop the loop with Ctrl+C.

Optional B - TLS at the edge

Concept: where does HTTPS end?

1. Termination at the edge (this lab)
   PC ==HTTPS==> HAProxy --HTTP--> Gateway --HTTP--> Pod
   + one place for certificates, the edge can read and filter HTTP
   - unencrypted inside the network

2. Passthrough (HAProxy in mode tcp, routes by SNI)
   PC ==HTTPS==================> Gateway --HTTP--> Pod
   + certificates live in the cluster (Secrets), managed by the platform
   - the edge sees only encrypted bytes: no HTTP rules, no X-Forwarded-For

3. Re-encryption
   PC ==HTTPS==> HAProxy ==HTTPS==> Gateway --> Pod
   + encrypted all the way   - two sets of certificates

All three are common. Termination at the edge is the simplest and fits our design: the edge already reads HTTP.

A certificate proves that the server owns the name. One wildcard certificate *.app.internal covers red., blue., green. and every future app - but only one level (a.b.app.internal is not covered). In a company it is issued by the internal CA; here we make a self-signed one and tell Windows to trust it.

# 31 - a self-signed wildcard certificate, valid for one year
mkdir -p ~/tls && cd ~/tls
openssl req -x509 -newkey rsa:2048 -sha256 -days 365 -nodes \
  -keyout app.internal.key -out app.internal.crt \
  -subj "/CN=*.app.internal" \
  -addext "subjectAltName=DNS:*.app.internal,DNS:app.internal"
openssl x509 -in app.internal.crt -noout -subject -ext subjectAltName -enddate

Browsers ignore the CN; they check the subjectAltName. A certificate without it is rejected even if the CN matches.

# 32 - HAProxy wants certificate and key in one file, readable only by root
sudo mkdir -p /etc/haproxy/certs
cat app.internal.crt app.internal.key | sudo tee /etc/haproxy/certs/app.internal.pem >/dev/null
sudo chmod 600 /etc/haproxy/certs/app.internal.pem
cd ~

Add an HTTPS frontend and redirect HTTP to HTTPS. Edit /etc/haproxy/haproxy.cfg (sudo nano /etc/haproxy/haproxy.cfg):

frontend apps_http
    bind 192.168.56.101:80
    http-request redirect scheme https code 301      # new: HTTP -> HTTPS
    default_backend k8s_gateway

frontend apps_https                                   # new frontend
    bind 192.168.56.101:443 ssl crt /etc/haproxy/certs/app.internal.pem
    http-request set-header X-Forwarded-Proto https
    http-request deny deny_status 404 unless { hdr_end(host) -i .app.internal }
    default_backend k8s_gateway

# 33 - validate, reload, test (-k: curl does not know our CA yet)
sudo haproxy -c -f /etc/haproxy/haproxy.cfg && sudo systemctl reload haproxy
curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' -H 'Host: red.app.internal' http://192.168.56.101/
curl -sk --resolve red.app.internal:443:192.168.56.101 https://red.app.internal/ | grep '"message"'

301 to https://red.app.internal/, then the red app over HTTPS.

On your PC, the browser warns about an untrusted certificate. Trust it (PowerShell as Administrator):

# 34 - copy the certificate (NOT the key) and trust it on this PC
scp student@192.168.56.101:tls/app.internal.crt $env:USERPROFILE\Downloads\
Import-Certificate -FilePath "$env:USERPROFILE\Downloads\app.internal.crt" `
  -CertStoreLocation Cert:\LocalMachine\Root

Restart the browser and open https://red.app.internal - a padlock, no warning. (Firefox has its own certificate store; accept the warning there or import the certificate in its settings.)

Warning

You just told Windows to trust a certificate you made yourself. That is fine in a lab and wrong everywhere else - remove it when you are done (see Clean up). The private key app.internal.key never leaves support01.

Optional C - Who is the client? X-Forwarded-For

Every proxy opens a new connection to the next hop. The app therefore sees the proxy’s address, not the user’s. HTTP solves it with a header that each proxy extends:

PC 192.168.56.1 -> HAProxy -> Traefik -> red Pod
                   adds       should keep it and add
                   X-Forwarded-For: 192.168.56.1

# 35 - what does the app see?
curl -s -H 'Host: red.app.internal' http://192.168.56.101/headers | grep -i -A2 forwarded

(Also open http://red.app.internal/headers in the browser on your PC.) X-Forwarded-For does not show 192.168.56.1: Traefik does not trust headers from unknown senders - anybody could send X-Forwarded-For: 1.2.3.4 - and replaces them with the address it sees. Tell Traefik which senders to trust: the cluster network (HAProxy and the nodes) and the Pod network (kube-proxy may forward via another node):

# 36 - trust forwarded headers from the edge and the cluster
cat > traefik-values-lb.yaml <<'YAML'
deployment:
  replicas: 2
service:
  type: LoadBalancer
  annotations:
    metallb.io/loadBalancerIPs: 192.168.128.150
ports:
  web:
    nodePort: 30080
    forwardedHeaders:                      # new
      trustedIPs: ["192.168.128.0/24", "10.244.0.0/16"]
  websecure:
    nodePort: 30443
providers:
  kubernetesGateway:
    enabled: true
gateway:
  listeners:
    web:
      namespacePolicy:
        from: All
YAML

The setting belongs to the entry point web - the door the requests come through.

# 37 -
helm upgrade traefik traefik/traefik -n traefik --version "$CHART_VERSION" \
  -f traefik-values-lb.yaml --wait
curl -s -H 'Host: red.app.internal' http://192.168.56.101/headers | grep -i -A2 forwarded

Now the first address in X-Forwarded-For is the original client. Trusting too much is a security hole (clients could fake their address), trusting too little loses the information - list exactly your proxies.

Discovery

Infodiscover

Task A. Stop the whole cluster from the edge’s point of view: what does the browser show when 192.168.128.150 does not answer? Where do you see it?

kubectl scale deployment traefik -n traefik --replicas 0
curl -s -o /dev/null -w '%{http_code}\n' -H 'Host: red.app.internal' http://192.168.56.101/
sudo tail -n 3 /var/log/haproxy.log
helm upgrade traefik traefik/traefik -n traefik --version "$CHART_VERSION" -f traefik-values-lb.yaml --wait

The ARP answer still comes (MetalLB announces while the Service exists), but nothing listens: HAProxy returns 503, the stats page shows the backend server red (DOWN), the log line shows <NOSRV>. The health check is what makes this a clean error page instead of a hanging browser.

Task B. A second pool: reserve 192.168.128.190-200 for databases only and make a Service ask for that pool explicitly.

Shrink lab-pool to 192.168.128.150-189, then:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata: { name: db-pool, namespace: metallb-system }
spec:
  addresses: ["192.168.128.190-192.168.128.200"]
  autoAssign: false           # only on request
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata: { name: db-l2, namespace: metallb-system }
spec:
  ipAddressPools: ["db-pool"]

The Service asks with the annotation metallb.io/address-pool: db-pool.

Task C. Why did we bind HAProxy to 192.168.56.101 and not to *?

* would also listen on 192.168.128.100, the cluster-side leg. An edge listens only where its clients are; everything else is attack surface.

Clean up

Nothing here breaks later labs - you may keep it. To undo everything:

# on support01
kubectl delete namespace colors
helm upgrade traefik traefik/traefik -n traefik --version "$CHART_VERSION" \
  -f traefik-values.yaml --wait                  # back to NodePort (Lab 14 values)
helm uninstall metallb -n metallb-system
kubectl delete namespace metallb-system
sudo systemctl disable --now haproxy
sudo cp /etc/haproxy/haproxy.cfg.orig /etc/haproxy/haproxy.cfg

# on your PC, PowerShell as Administrator
(Get-Content "$env:windir\System32\drivers\etc\hosts") -notmatch 'app\.internal' |
  Set-Content "$env:windir\System32\drivers\etc\hosts"
Get-ChildItem Cert:\LocalMachine\Root | Where-Object Subject -eq 'CN=*.app.internal' | Remove-Item
ipconfig /flushdns

If traefik-values.yaml from Lab 14 is gone, recreate it from Lab 14 step 5.

References