The question this answers
How does a workload in a private subnet call an external API without becoming reachable from the internet?
The billing service must call Stripe, pull container images and fetch OS security updates. It must never accept an inbound connection from the internet.
Outbound-initiated connectivity with no inbound path: return traffic for connections the workload opened is allowed back, and nothing else gets in.
Outbound is not the same permission as inbound
A private subnet is defined by what its route table does *not* contain: no route to an internet gateway. That is a strong default — an instance there cannot be scanned, cannot be brute-forced on port 22, and cannot be reached by a botnet probing for exposed databases. It is also, on its own, useless for most real workloads, because almost everything needs to call *out*: payment APIs, image registries, package mirrors, OS updates, telemetry endpoints.
A NAT device resolves the asymmetry. It sits in a public subnet, holds a public address, and performs source network address translation on behalf of workloads that have none. The private workload opens a connection; the NAT rewrites the source address to its own and remembers the mapping; the reply comes back to the NAT and is translated back. Because the translation table is only ever populated by *outbound* connections, there is no entry an inbound packet could match. The one-way property is a consequence of how the state table is built, not a firewall rule someone remembered to add.
This is the same mechanism the Networking domain teaches as NAT — see NAT Gateway for the cloud framing and the Networking domain for the packet-level translation. What is specific to cloud is that the NAT is a *billed, capacity-limited, per-zone* resource rather than a feature of your router.
The failure nobody predicts: port exhaustion
A NAT device multiplexes every private workload behind one address, and each concurrent connection to the same destination consumes one source port from a finite range. The limit is per destination address and port pair, and it is in the tens of thousands. A fleet that opens a fresh TLS connection per outbound request — no keep-alive, no pooling — to a single popular endpoint can walk into that ceiling under load.
What makes this a genuinely nasty incident is where it does *not* show up. The application health check runs inside the virtual network and never traverses the NAT, so it stays green. CPU is fine, memory is fine, the load balancer reports every target healthy. What the user sees is that checkout hangs and then fails, because the outbound call to the payment provider could not allocate a port. Every dashboard says the service is up.
The fix is almost never a bigger NAT. It is connection reuse — an HTTP client with a warm keep-alive pool turns thousands of short-lived connections into a handful of long-lived ones — plus a private endpoint for the destinations that support one, which removes those flows from the NAT path entirely.
- Source-port allocation is per
(destination address, destination port)— one hot endpoint exhausts long before total connection counts look alarming. - Connections in
TIME_WAITstill hold their mapping for the timeout window, so the effective ceiling is lower than the raw port count. - A NAT device lives in one availability zone. One per zone, or a zone failure silently removes egress for every workload routed through it.
It is also one of the most surprising line items
NAT pricing has two meters: an hourly charge for the device existing, and a per-gigabyte charge for everything processed through it. The second one is what surprises teams, because it applies to traffic that feels internal. Pulling a multi-gigabyte container image on every deploy, shipping logs to a third-party endpoint, or reading from object storage *without* a private endpoint all bill per gigabyte through the NAT — and the object-storage case is the classic one, because the data never needed to leave the provider network at all.
The lesson generalizes past NAT: in cloud infrastructure, the component that moves bytes is frequently more expensive than the component that stores or computes on them. See Egress: Moving Data Costs Money, Not Just Storing It.
Bars are relative weights, not currency. Real rates depend on provider, region, commitment and volume.
Key points
- A NAT gives outbound connectivity without inbound reachability, because its translation table is only ever populated by outbound connections.
- It is a per-zone resource: one NAT means one zone's failure removes egress for everything routed through it.
- Source-port exhaustion is the signature failure, and internal health checks stay green throughout it.
- It bills per gigabyte processed, which quietly taxes image pulls and traffic to provider services that a private endpoint would carry for free.
The loop, answered
Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.
- • The private subnet's route table sends
0.0.0.0/0to the NAT device rather than to an internet gateway. - • The workload opens a connection; the NAT rewrites the source address to its own public address and allocates a source port, recording the mapping.
- • The internet gateway forwards the translated packet out, since the NAT — unlike the workload — has a public address.
- • Return traffic matches the recorded mapping and is translated back to the private address. Traffic matching no mapping is dropped, which is the whole security property.
- • Deploy one NAT per availability zone and route each zone's private subnets to their own — this is the single most commonly skipped step.
- • Watch port-allocation and dropped-connection metrics, not just throughput; throughput looks healthy right up to exhaustion.
- • Own the egress allow-list: a NAT permits outbound to anywhere by default, which is exactly the path data exfiltration uses.
- • Route provider-service traffic through private endpoints so it never reaches the NAT meter or the NAT's capacity.
- • Source-port exhaustion under load: outbound calls hang and time out while every internal health check reports healthy.
- • Zone failure with a single NAT: workloads in surviving zones lose egress even though they are running fine.
- • Idle-timeout resets on long-lived connections that send no traffic — commonly seen as a database or message-broker connection that dies after minutes of quiet.
- • A missing
0.0.0.0/0route, which presents identically to a firewall problem: connection attempts hang rather than being refused.
- • Throughput scales without intervention on managed NAT offerings; the ceiling you actually hit first is concurrent source ports to a single hot destination.
- • Adding instances multiplies outbound connections linearly, so a fleet that doubles doubles its port consumption — connection pooling changes the slope, more capacity does not.
- • Private endpoints scale better than a bigger NAT because they remove flows from the path rather than widening it.
- • The trust boundary is one-directional by construction: outbound-initiated only, with no inbound path to translate.
- • It is not egress *control*. Default NAT rules allow outbound to any destination, so a compromised workload has a working exfiltration channel.
- • Add an egress firewall or proxy with an allow-list when the workload handles regulated data — see Egress Security and Network Segmentation.
- • NAT logs are the record of what your private workloads talked to. Retain them; they are frequently the only evidence in an exfiltration investigation.
- • Two meters: hourly device existence, and per-gigabyte data processed.
- • Zone redundancy multiplies the hourly meter by the number of zones — a real cost of doing reliability correctly.
- • The per-gigabyte meter catches image pulls, log shipping and object-storage reads that a private endpoint would have carried without touching it.
- • Port-allocation errors and dropped-connection counts — the leading indicator of exhaustion.
- • Bytes processed, split by destination, which is simultaneously the cost signal and the exfiltration signal.
- • Per-zone NAT health, so you find out that a zone lost egress before your users do.
- • The signal that lies: application health checks and load-balancer target health, both of which stay green through a total egress outage because neither traverses the NAT.
- • A private endpoint for provider services — cheaper, faster and it removes the flows from the NAT entirely. Do this first for object storage.
- • Put the workload in a public subnet with a strict inbound security group when it genuinely needs to accept inbound traffic anyway; a NAT in front of something already public buys nothing.
- • An egress proxy with an allow-list when you need outbound *control* rather than outbound *connectivity* — NAT does not give you the former.
- • For a workload with no outbound needs at all, no egress path is the correct and cheapest answer. Not every private subnet needs a NAT.
- • Buys a strong one-directional boundary; costs a metered dependency on the path of every external call your system makes.
- • Zone-redundant NAT multiplies a fixed cost to remove a single point of failure — a genuine reliability-versus-cost decision, not an obvious one.
- • It adds a hop that is invisible to application-level monitoring, which is exactly why its failures are diagnosed late.
Outbound only — and then the ports run out
translation table (0 entries) private source nat source destination —
What people believe, and what is true
A NAT gateway is a firewall.
It blocks unsolicited inbound as a side effect of its state table, and permits all outbound. It is not egress control and has no allow-list.
Everything in a private subnet needs a NAT.
Only workloads that must initiate outbound connections. A worker that reads a queue and writes a database over private endpoints needs none.
NAT throughput is the thing that runs out.
Managed NAT scales throughput. Source ports to one hot destination run out first, and the symptom looks nothing like a bandwidth problem.