The question this answers
What actually happens between an event arriving and my code running, when there is no instance I provisioned?
Thumbnail generation runs a few hundred times an hour, in bursts that follow user uploads, and does nothing at 03:00. Nobody on the team wants to patch a VM that is idle 90% of the day.
Per-request allocation: an execution environment that exists because an event arrived, scales with the arrival rate without a scaling policy, and bills nothing while no events arrive.
The chain: event → platform → environment → execution → reclaim
Serverless is best understood as a change in *what you allocate*. With a VM you allocate a machine and then find work for it. With a container platform you allocate a replica count and then route work to it. With serverless you allocate nothing; you register code and a trigger, and the platform allocates an execution environment at the moment an event needs one.
That inversion is the whole model. Concurrency is not a number you set — it is the number of events currently in flight. Capacity planning becomes rate-limit planning. And the scale-*down* step, which every other model treats as an optimization, is here a guarantee: the environment goes away, taking its memory, its open sockets and any state you left in a module-level variable with it.
The trigger matters as much as the code. An HTTP request through an API gateway, a message on a queue, an object landing in a bucket and a cron schedule are all the same shape to the platform — something happened, run this — but they differ enormously in retry behaviour, and that is where most production surprises live. See Serverless Trade-offs.
- 1Event arrivesms
A request, queue message, object-created notification or timer fires and is handed to the platform's front door.
The trigger's own limits apply first: gateway timeouts and queue visibility windows cap what the function may take.
- 2Platform routes itms
The platform finds a warm environment for this version of this function, or decides to create one.
Concurrency limit reached — the platform throttles rather than queues, and the caller sees a rejection.
- 3Environment initialized100s of ms to seconds
Runtime starts, deployment package is loaded, module-level code runs, connections and clients are constructed.
This is the cold start. Heavy dependencies and eager connection setup are paid here, on a user-facing request.
- 4Handler executesbounded by the configured limit
Your function body runs with the event payload and an execution context carrying a deadline and an identity.
Hard wall-clock timeout — the platform kills the execution mid-work with no unwind.
- 5Environment frozenminutes, undocumented
The environment is kept warm for reuse. Background work not awaited before returning is suspended, often invisibly.
Fire-and-forget work "completes" minutes later inside an unrelated invocation, or never.
- 6Environment reclaimedprovider-decided
The platform destroys the environment. Memory, cached data and open connections go with it.
Anything treated as a cache or a session store silently loses entries; downstream connection counts drop without notice.
What "managed runtime" actually moves off your plate
The honest framing is not "no servers" but "a much higher boundary". The provider takes the host, the hypervisor, the operating system, the runtime patching and the placement of your code onto capacity. You keep the code, its dependencies, its configuration, its identity, its network placement and the shape of the bill — which is exactly the set of things that cause incidents.
This is the Shared Responsibility line drawn further up than anywhere else in the domain. It is genuinely a large reduction in operational work: no base image to rebuild, no kernel CVE to chase, no autoscaling group to tune. It is not the elimination of operations, and a team that believes it is will discover the gap the first time a dependency upgrade changes cold-start time or the platform deprecates a runtime version on its own schedule, not yours.
Where it fits the workload, and where it fights it
The workloads serverless suits are the ones whose shape matches the allocation unit: spiky, short, independent, and driven by an event that already exists. Image processing after upload, webhook receivers, scheduled reports, glue between two managed services, and the fan-out step of a pipeline are all natural fits because the platform's unit — one event, one bounded execution — is also the workload's unit.
The workloads it fights are the ones that want to *hold* something: a long-lived connection pool, a warm in-process cache, a websocket, a job that runs for twenty minutes, or a request path where a 400 ms cold start is unacceptable. None of those are impossible on serverless, but each one is worked around rather than served, and the workarounds are where the complexity you thought you avoided reappears.
Note also the topology consequence: a function placed inside your virtual network can reach a private database, but then it needs the same egress plumbing as any other private workload — a NAT Gateway or a private endpoint — and it inherits that component's failure modes and bill.
Key points
- Serverless changes the unit of allocation from a machine or a replica count to a single event-driven execution.
- Concurrency is an observed consequence of arrival rate, not a number you configure — which is why the ceiling you hit is a throttle, not a queue.
- The reclaim step is a guarantee, not an optimization: anything held in process memory or in an open connection is expected to vanish.
- The provider takes the host, OS and runtime patching; you keep code, dependencies, identity, network placement and the bill.
- It fits spiky, short, independent, event-shaped work — and fights anything that wants to hold state or a connection.
The loop, answered
Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.
- • You register a deployment package plus a trigger; nothing runs and nothing bills until the trigger fires.
- • An event arrives at the platform's front door, which applies the trigger's own limits before your code is considered.
- • The platform reuses a warm execution environment for that function version, or creates one — runtime start plus module-level initialization.
- • The handler runs with an event payload, a deadline and an execution identity the platform injects rather than a credential you shipped.
- • On return the environment is frozen for possible reuse, and eventually reclaimed along with everything it held.
- • Own the deployment package: dependency size directly buys or costs you cold-start latency on every scale-out.
- • Own the execution identity — the function's role is the blast radius of any dependency compromise, and the default templates are usually too broad.
- • Own retry semantics per trigger: synchronous triggers push failure to the caller, asynchronous ones retry silently, queue triggers retry until a dead-letter destination catches them.
- • Own runtime upgrades on the provider's deprecation calendar, and test them, because the runtime is patched underneath you but the version is not.
- • Own the concurrency ceiling and its distribution across functions, or one noisy function throttles the account.
- • Throttling under a burst: the platform rejects rather than queues, so a synchronous caller sees errors while every function-level metric looks healthy.
- • A hard timeout kills a long execution mid-write, leaving partially applied work with no unwind path — the reason idempotency matters more here than anywhere.
- • Async triggers retry an already-succeeded-but-slow invocation, producing duplicates in the downstream system.
- • Work started but not awaited before returning is frozen with the environment and completes minutes later, inside a different request, or not at all.
- • A runtime deprecation forces an unplanned migration with a deadline set by the provider.
- • Scales with arrival rate automatically and without a policy; the first ceiling is the account or function concurrency limit, not CPU.
- • The second ceiling is always downstream: the database connection pool, a third-party API rate limit, or a partner's quota. See Serverless and Database Connections.
- • Burst scaling is rate-limited by the platform — going from 10 to 3000 concurrent executions is not instantaneous, and the ramp shows up as elevated latency at the start of a spike.
- • Scale-to-zero is the other half: the first request after a quiet period pays a cold start, so low-traffic endpoints are consistently the slowest ones.
- • The function has a platform-issued identity rather than a static key — a genuine improvement over credentials baked into an image. See Roles vs Static Keys.
- • Blast radius is the execution role, and the default is usually broader than the handler needs; scope it to the specific resources it touches.
- • The event payload is untrusted input. A function triggered by an object upload is triggered by whatever a user chose to upload.
- • A function outside your virtual network reaches managed services over the provider's network; inside it, it needs subnet placement, security groups and an egress path like any other workload.
- • Two meters: number of invocations, and allocated-memory × execution duration. Idle costs nothing, which is the whole appeal.
- • Memory size sets both the price per millisecond and the CPU share, so a larger setting is sometimes cheaper because the function finishes proportionally faster.
- • Everything the function calls has its own meter — gateway requests, queue operations, log ingestion and data transfer — and the sum of those often exceeds the compute line.
- • The shape is linear in traffic with no floor, which inverts the usual cloud problem: there is no idle waste, and no volume discount either.
- • Invocation count, error rate and throttle count — the third is the one that reveals a concurrency ceiling and the one nobody graphs.
- • Duration distribution split by cold and warm, because the mean hides the tail that users actually experience.
- • Dead-letter queue depth, which is the only place a silently retried and permanently failed async event shows up.
- • The signal that lies: average duration. It is dominated by warm invocations and stays flat through a cold-start regression that doubles p99.
- • A single small always-on instance running a worker loop, when traffic is steady — it is cheaper, has no cold start, and holds a connection pool without a proxy.
- • A managed container service that scales to zero, when the workload wants a normal HTTP server and a longer execution budget but the same "no machines" property.
- • A cron job on an instance you already run, when the requirement is genuinely just "do this at 03:00" — adding an event platform for one scheduled task is No Cargo-Cult Infrastructure material.
- • The provider's built-in integration, when the function only copies an event from one managed service to another; that glue frequently exists already and needs no code.
- • Buys elimination of instance and OS management; costs you control over start-up latency, execution duration and where your process lives.
- • Buys automatic scaling; costs you the ability to hold anything — pools, caches, sockets — across invocations.
- • Buys per-use billing; costs predictability, because the bill now tracks traffic exactly, including traffic you did not want.
- • Buys a smaller operational surface; costs portability, since triggers, packaging and identity are the most provider-shaped parts of the cloud.
What people believe, and what is true
Serverless means there are no servers.
There are servers; you do not choose, patch or scale them. The boundary moved up — it did not disappear, and everything above it is still yours.
Serverless removes capacity planning.
It replaces instance planning with concurrency and quota planning. The account limit, the burst ramp and every downstream rate limit are still capacity decisions.
Functions are stateless, so state is not a concern.
The platform *enforces* statelessness between reclaimed environments but happily reuses a warm one, so leaked state produces bugs that only appear under reuse.