KubernetesKUBERNETES-SPECIFICTOOL-SPECIFIC

Getting Traffic Into the Cluster

Internal Services are unreachable from outside. Something at the edge must terminate TLS, match hostnames and paths, and route to the right Service — and which object expresses that is currently in transition.

What actually happens How to do it properly Evidence and rollback

The question, the obvious approach, and why it breaks

Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.

The production question

How does a request from the internet reach the right workload, and who owns the rules that decide?

The problem

A cluster full of ClusterIP Services has no front door. Giving each workload its own cloud load balancer works and multiplies cost, certificates and configuration by the number of services.

What teams do first

Give every externally reachable Service type: LoadBalancer. It works immediately, each service gets an address, and there is nothing new to operate.

How it breaks

Cost scales with service count. Each cloud load balancer is billed independently whether it carries one request a day or a million (Cost Drivers).

How it breaks in production

Cost scales with service count. Each cloud load balancer is billed independently whether it carries one request a day or a million (Cost Drivers).
Certificates multiply. Every external address needs its own certificate and its own renewal, and renewal is one of the most reliable sources of avoidable outages (Renewal: Automating the Thing That Expires).
Path-based routing becomes impossible. If /api and /checkout on one hostname must reach different workloads, per-service load balancers cannot express it.
Cross-cutting edge concerns — redirects, headers, rate limits, request logging — have to be implemented once per service instead of once at the edge (API Gateway is the architecture-level treatment).

Code→Build→Test→Artifact→Release→Deploy→Run→Observe→Operate→Incident→Recover→Learn→Improve

What is actually happening

Underneath the tooling, which is the part that survives a change of tool.

Two things are always involved and are constantly confused. The rules are an API object; the implementation is a controller plus its proxy pods running in the cluster. Creating the rules does nothing unless a controller is watching for them.
The controller watches routing objects, generates its proxy configuration, and reloads. Traffic arrives at that proxy — usually through one cloud load balancer for the whole cluster — and the proxy matches host and path, terminates TLS, and forwards to the target Service (Services: A Stable Address Over Moving Pods).
Ingress is the long-standing object: HTTP and HTTPS only, host and path rules, TLS referencing a Secret, and an ingressClassName naming which controller should act on it.
Because Ingress expresses so little, implementations extended it with annotations — timeouts, rewrites, auth, rate limits — which are controller-specific and do not port between implementations. This is the practical reason ingress configuration is so hard to move.
The Gateway API is the successor: a separate set of objects where a Gateway describes listeners and a HTTPRoute describes matching and backends, deliberately splitting platform-owned edge configuration from team-owned routes. It also covers protocols beyond HTTP.
Both are just rules. The failure modes belong to the controller, the proxy, the certificate and the underlying load balancer — which is why "the ingress is broken" is almost never a statement about the object you edited (Operating the Edge).

The same routing, in both object models

Reading these side by side is the fastest way to see what Gateway API is actually for. The Ingress mixes listener configuration, TLS and routes into one object owned by one team. Gateway API splits them: a platform team owns the Gateway and its certificate, an application team owns the HTTPRoute that attaches to it.

Ingress (left) and Gateway API (right), routing /api and /checkout

1# --- Ingress: rules, TLS and listener in one object ---
2apiVersion: networking.k8s.io/v1
3kind: Ingress
4metadata:
5  name: shop
6spec:
7  ingressClassName: nginx        # names the controller that must act on this
8  tls:
9    - hosts: ["shop.example.com"]
10      secretName: shop-tls       # a Secret holding the certificate and key
11  rules:
12    - host: shop.example.com
13      http:
14        paths:
15          - path: /api
16            pathType: Prefix
17            backend:
18              service:
19                name: api
20                port:
21                  number: 80
22 
23# --- Gateway API: listener and route are separate objects ---
24apiVersion: gateway.networking.k8s.io/v1
25kind: Gateway
26metadata:
27  name: shop-edge              # platform-owned: listeners and certificates
28spec:
29  gatewayClassName: example-gw
30  listeners:
31    - name: https
32      protocol: HTTPS
33      port: 443
34      hostname: shop.example.com
35      tls:
36        certificateRefs:
37          - name: shop-tls
38---
39apiVersion: gateway.networking.k8s.io/v1
40kind: HTTPRoute
41metadata:
42  name: checkout-route         # team-owned: just this team's paths
43spec:
44  parentRefs:
45    - name: shop-edge
46  hostnames: ["shop.example.com"]
47  rules:
48    - matches:
49        - path:
50            type: PathPrefix
51            value: /checkout
52      backendRefs:
53        - name: checkout
54          port: 80

The split is the point. With Ingress, every team that needs a path edits the object that also holds everyone's TLS configuration. With Gateway API, a route is a separate object that attaches to a Gateway, so a team can add a path without being able to break the listener.

Where an external request actually goes

Five hops, each owned by someone different, each able to fail on its own. Naming the hop is most of the diagnosis when external traffic fails and every pod looks healthy.

Internet to pod

UserLLMAgentToolDataDecisionHumanGuardrail

Edge failures and where to look first

KUBERNETES-SPECIFICRows two, four and five happen on every platform — expired certificates and mismatched timeouts are universal. Rows one and six are specific to a routing layer that is itself a workload in the cluster: on a managed API gateway there is no controller to be unscheduled, and no class name to get wrong.

These share a symptom — the workload is healthy and users cannot reach it — and have entirely different causes. The response column is the first move, not the fix.

Trigger	Symptom	Cause	Response
Rules applied, nothing happens	No external address is ever assigned	No controller is watching that class, or the class name is wrong	Check the controller is running and the class matches; the object alone does nothing
Certificate expires	Every host behind the edge fails TLS at once	Renewal automation failed silently and nothing alerted	Replace the certificate; then alert on days-to-expiry, not on renewal success (Renewal: Automating the Thing That Expires)
Path rule shadows another	Some URLs reach the wrong workload; all health checks pass	Prefix match on a broader path takes precedence over the intended rule	Read the generated proxy configuration, not the object you wrote
502 from the edge	Edge returns errors; backend pods look healthy	Service has no ready endpoints, or targetPort mismatch behind it	Check the Service's endpoint list before touching the edge (Services: A Stable Address Over Moving Pods)
Slow requests truncated	Long requests fail at a consistent duration	Proxy timeout shorter than the backend's, or shorter than the cloud load balancer's	Align timeouts across all three hops, outermost longest (Timeouts: The Latency Contract Nobody Writes Down)
Controller pods evicted	Total external outage; cluster internals fine	Edge proxies had no resource requests and lost a scheduling contest (Requests and Limits)	Give edge components guaranteed resources and spread them across nodes

How to do it properly

Most important first.

Run one shared entry point for HTTP traffic and route by host and path, rather than one load balancer per service.
Automate certificate issuance and renewal, and alert on approaching expiry rather than relying on the automation being silent about failure (Certificates as an Operational Object).
Treat routing rules as production changes with their own blast radius. A bad path rule at the edge takes out every workload behind that hostname, instantly, with no rollout to stop (Blast Radius: If This Is Wrong, How Much Does It Affect?).
Know which implementation you run and stay aware of how much of your configuration lives in its annotations, because that is exactly the part that will not port.
Split ownership if you can: the platform owns listeners, certificates and edge policy; teams own the routes to their own Services. That split is what Gateway API encodes, and it is worth adopting as a convention even on plain Ingress.
Give the edge its own signals — request rate, error rate and latency at the ingress, separately from each backend — because the edge can be the problem (The Four Golden Signals).

How much can this affect

Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.

Blast radius if this is wrongEveryone

One testEveryone

What contains it

Edge rules apply to all external traffic the moment they are accepted; the only real containment is validation before apply and a second entry point for genuinely critical paths.

What can go wrong

Failure modes, including of the mitigation

Rules created with no controller watching that class, so nothing happens at all and there is no error anywhere.
A certificate expires. Every hostname behind that entry point fails at once for every client, and the failure is total rather than partial (Renewal: Automating the Thing That Expires).
A path rule with the wrong match type shadows a more specific rule, sending a subset of traffic to the wrong Service. Health checks all pass.
The ingress controller's own pods are unhealthy or under-provisioned, so the cluster is fine and nothing reaches it (Operating a Load Balancer).
Timeout mismatch between the cloud load balancer, the proxy and the backend, producing truncated responses on slow requests that nobody can reproduce (Timeouts: The Latency Contract Nobody Writes Down).
Annotation-driven behaviour lost during a controller migration, so a rewrite or auth rule silently stops applying while everything still returns 200.

Misreads this invites

"An Ingress is a load balancer." It is a set of rules. The load balancer is the controller's proxy plus whatever cloud load balancer sits in front of it.
"Creating the object exposes the service." Only if a controller for that class is running and watching. Otherwise it is an inert record of intent.
"Gateway API replaces Services." It replaces the routing layer above them. Backends are still Services (Services: A Stable Address Over Moving Pods).
"Ingress configuration is portable." The object is; the annotations that make it behave the way you need are not.
"The edge is infrastructure, so it is not a deploy." It is one of the highest-blast-radius changes you can make, with no canary and no rollout (Change Size: Why Small Changes Are Safer, and When They Are Not).

Operating it

Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.

How you know it worked

A request from outside reaches the intended Service — verified end to end, not inferred from the object existing.
The routing object reports an assigned address and the controller reports it as accepted.
Certificate expiry dates are known, monitored and further away than the renewal interval.
Edge request and error rates are visible per host and path, so you can tell an edge problem from a backend problem in one look (Dashboards an Operator Can Act On).

How you get back

Routing objects are declarative and revert instantly by reapplying the previous version — but "instantly" cuts both ways: there is no gradual rollout, so the bad state was also instant and total.
Certificate problems do not roll back. An expired certificate can only be replaced, which is why the automation and its alerting are the real control (Renewal: Automating the Thing That Expires).
Changing ingress controller is a migration, not a rollback. Keep the old one serving until the new one is verified for every host.

What to automate, and what stays human

Automate certificate issuance, renewal and the alert on renewal failure. This is the highest-value automation at the edge.
Automate validation of routing rules before apply: conflicting hosts, overlapping paths and references to Services that do not exist (Policy as Code).
Keep hostname changes and public exposure decisions human. Exposing something publicly is a security decision, not a routing detail (Public Exposure, Read With Context).

What this costs

One shared entry point is cheaper and simpler and is a shared failure domain: its outage is everyone's outage, and its configuration is a shared object several teams change.
Ingress is universally supported and expresses little, so real deployments depend on controller-specific annotations. Gateway API expresses far more and is a newer object model with its own migration cost.
Terminating TLS at the edge simplifies backends and means traffic inside the cluster is unencrypted unless you do something about it (Encryption at Rest vs in Transit).

Where this applies

This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.

KUBERNETES-SPECIFICIngress and Gateway API are Kubernetes objects. Outside a cluster the same job is done by a cloud load balancer with listener rules, an API gateway product, or an nginx configuration file on a VM — all of which are edited directly rather than reconciled from a declared object, so they drift instead of being reasserted.
TOOL-SPECIFICBehaviour beyond host and path matching depends on the controller — ingress-nginx, Traefik, HAProxy, Envoy-based controllers, or a cloud provider's. Timeouts, rewrites, header handling and rate limiting differ, and are usually expressed as annotations that do not port between them.

Where the depth lives

This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.

Cloudingress-and-gateway cloud-load-balancers public-exposure

Securitytls-as-a-boundary

OS & Networkingtls-handshake certificates proxies

Architectureapi-gateway