Serverless Backends
A per-invocation execution model that removes supervision and adds cold starts, execution limits and connection pressure — excellent for some workloads and wrong for others.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.
What changes about writing a backend when the platform creates and destroys the process around each request?
Traffic is spiky and mostly zero. Nobody wants to operate instances for a webhook receiver and three internal endpoints.
Take the existing service, wrap each route in a function handler, and deploy. Same code, no servers to manage, and we only pay when it runs.
Each concurrent invocation opens its own database connection, so a burst of 300 concurrent requests attempts 300 connections against a database configured for 100 and every one of them fails (Connection Pools).
- Each concurrent invocation opens its own database connection, so a burst of 300 concurrent requests attempts 300 connections against a database configured for 100 and every one of them fails (Connection Pools).
- The first request after an idle period pays initialisation — runtime start, dependency loading, connection establishment — and the user sees it as a slow request, not as a platform detail (Startup Time & Cold Start in Cloud & Infrastructure).
- A report endpoint exceeds the platform's maximum execution duration and is terminated mid-work, with a partial write already committed.
- Work started after the response — a fire-and-forget analytics call, a queued follow-up — never runs, because the execution environment is frozen or destroyed once the response is returned.
- An in-process cache has no effect, because the process it lives in serves one request and may not exist for the next.
- Costs are dominated by a high-volume endpoint that runs constantly, where the per-invocation model is a poor fit for sustained throughput.
What is actually happening
- The platform owns the process lifetime. It creates execution environments on demand, may reuse a warm one for a subsequent invocation, and destroys them on its own schedule. Your code has no say in any of it.
- Concurrency model is the property that matters most, and it differs between platforms: some run exactly one invocation per environment at a time, so N concurrent requests means N environments and N of everything the handler opens; others serve many concurrent requests per instance, closer to a normal server (Mapping Services Across Cloud Providers).
- Cold start is the initialisation of a new environment: runtime boot, code load, module initialisation, and whatever your handler does on first call. It is on the critical path of the request that triggers it, and its magnitude depends on runtime, package size and initialisation work — not on a number anyone can quote for you.
- Between invocations a reused environment may be frozen: background timers, open sockets and in-flight promises are suspended, and may resume much later or never. Work not completed before the response is not reliably completed at all.
- There is a maximum execution duration, and it is a hard stop. Long jobs must be decomposed or moved to a different compute model (Background Jobs).
- Statelessness is enforced rather than encouraged: there is no reliable local disk, no shared memory across invocations, and no guarantee that two requests from one user touch the same environment (Stateless Services).
- Scaling is per-invocation and effectively immediate, which means the platform can present your downstream dependencies with a burst of concurrency they were never sized for. The function scales; the database does not.
Where the model genuinely wins
The honest case for serverless is not cost and not scale — it is operational shape. Some workloads are intermittent, event-shaped and independently deployable, and running a permanently-alive instance for them is pure overhead in both money and attention.
The case against is equally specific and is mostly about coupling to state: anything that wants a warm process, a shared pool, a long execution or a predictable tail latency is fighting the model rather than using it.
What shape is this piece of work?
when Bursty, low average volume, must verify and enqueue quickly.
cost Cold start on the first event after quiet; the sender may have a short timeout (Inbound Webhooks).
when React to an object upload, a queue message, a schedule.
cost At-least-once delivery means every handler must be idempotent.
when Admin tools, internal APIs with a handful of users.
cost Latency is nobody's problem here, which is exactly why it fits.
when Traffic swings by orders of magnitude and is often near zero.
cost Connection strategy and concurrency caps become mandatory design work.
when Constant load, latency-sensitive, heavy database use.
cost Poor fit: warm capacity negates scale-to-zero, and connection pressure is worst here.
when Anything beyond the platform's duration limit.
cost Does not fit at all; decompose into steps or use a container/worker model (Background Jobs).
when WebSockets, SSE, long-poll.
cost Needs a platform-specific connection service; the plain function model does not hold connections.
The connection problem, in arithmetic
This is the failure that ends most first serverless projects, and it is not subtle once written down. On a platform where each execution environment handles one request at a time, concurrency and process count are the same number. Every process that opens a database connection means one connection per concurrent request — with none of the sharing that a pool inside a long-lived server provides.
A fixed fleet degrades gracefully here: requests queue for a pool slot and latency rises. A per-invocation platform does not queue — it creates more environments, each of which opens more connections, until the database refuses them. The mitigation is a pooler or proxy between the functions and the database, an HTTP-based data API, or a hard cap on concurrency.
// module scope — reused if the environment is warm
const pool = new Pool({ max: 10 })
export async function handler(event) {
const c = await pool.connect()
try { return await work(c, event) }
finally { c.release() }
}
// 1 env = 1 concurrent request (on this model)
// 300 concurrent requests -> 300 environments
// -> each may open connections, up to max: 10
// -> the database sees hundreds of connections
// for a workload of 300 in-flight queries
// Database max_connections: 100. Everything fails.// One connection per environment, at most.
const pool = new Pool({ max: 1, idleTimeoutMillis: 5_000 })
// ...and a pooler between functions and the database:
//
// functions (N environments)
// | many short-lived client connections
// v
// connection proxy / pooler <-- multiplexes
// | a small, fixed number of server connections
// v
// Postgres (max_connections: 100)
//
// Plus a platform concurrency cap so N is bounded,
// and the burst is throttled rather than amplified.A connection pool is an optimisation for a process that outlives many requests. In a per-invocation model the process does not, so the pool degenerates into "one or more connections per concurrent request" — it multiplies the problem instead of solving it. The multiplexing has to happen somewhere that *is* long-lived, which means outside your function. Capping concurrency matters just as much: without it, a traffic spike is converted directly into connection pressure.
The invocation lifecycle, and where things go wrong
Most serverless surprises come from one assumption carried over from long-lived servers: that the process continues after the response. It does not reliably continue, and reasoning about the lifecycle explicitly removes an entire class of bug.
Note where cold start sits. It is not a background cost amortised over the service — it is charged to one unlucky request, which is why it shows up in the tail and never in the average.
- 1Environment created
Platform provisions an execution environment.
fails by Concurrency limit reached: the invocation is throttled rather than queued.
- 2Runtime + code load
Runtime boots, package is loaded.
fails by Large deployment package lengthening every cold start.
- 3Module init
Module-scope code runs: clients, config, secrets.
fails by Expensive work here is paid on every cold start; secrets fetched per invocation instead of per environment.
- 4Handler runs
Your request logic, with a hard duration ceiling.
fails by Timeout truncation mid-transaction, with no error path executed.
- 5Response returned
Result sent to the caller or event source.
fails by Assuming deferred work will continue — it may be frozen here.
- 6Telemetry flush
Logs, metrics and spans must be flushed before returning.
fails by Freeze mid-flush: the failure you most want to see has no trace.
- 7Environment frozen
Kept warm for a while; module state persists.
fails by Per-user data cached at module scope leaking to the next invocation.
- 8Environment destroyed
Reclaimed on the platform's schedule.
fails by Connections dropped rather than closed, leaving the database to time them out.
How to build it
Most important first.
- Initialise expensive things outside the handler, at module scope, so a reused environment pays for them once. Clients, config parsing and compiled schemas belong there; per-request state does not.
- Solve the connection problem explicitly. Either put a connection proxy or pooler in front of the database, use a data API that speaks HTTP rather than a database wire protocol, or cap the platform's concurrency so the arithmetic works (Connection Pools).
- Bound concurrency deliberately. Reserved or maximum concurrency is the throttle that protects everything downstream from a traffic spike (Bounding Concurrency in Concurrency & Parallelism).
- Finish everything before returning the response, or hand the work to a queue and let a separate invocation do it. Never rely on work continuing after the response (Job Queues).
- Keep deployment packages small and initialisation cheap — package size and module-level work are the parts of cold start you control.
- Choose the workload deliberately: event-driven glue, webhook receivers, scheduled tasks, sporadic internal endpoints and bursty asynchronous processing are natural fits (Inbound Webhooks).
- For latency-sensitive paths, either keep instances warm through the platform's provisioned-concurrency mechanism or use a model that keeps a process alive — and be aware that keeping capacity warm removes the scale-to-zero property that made the model attractive.
- Design for at-least-once everywhere. Event sources retry, and a retried invocation is a duplicate execution (Idempotency in Backends).
What can go wrong
- Connection exhaustion at the database under a burst — the classic serverless incident, and it is caused by success rather than by a defect.
- Timeout truncation mid-transaction, leaving a partial effect with no error handler having run.
- Retries by the event source on a handler that is not idempotent, producing duplicate side effects (At-Least-Once Delivery).
- A cold-start latency spike that appears only in the tail, invisible in averages, and worst for the least-used endpoints (Percentiles: Which One, and How Many Users Is That? in Observability & Performance).
- Traces that stop abruptly, because the exporter was still flushing when the environment was frozen. Telemetry must be flushed before returning.
- Local development that behaves nothing like production, because the concurrency, timeout and freeze semantics have no local equivalent.
- A slow downstream dependency multiplying concurrency: each invocation waits, so more invocations exist simultaneously, so more connections are opened — the opposite of the queueing behaviour a fixed-size fleet would have shown (Little's Law as Working Intuition in Observability & Performance).
- An event source retry can start a second invocation while the first is still running, so two executions of the same event overlap. Idempotency is not optional in this model (Duplicate Detection).
- Module-scope state in a reused environment is shared between consecutive invocations, and with concurrent ones on platforms that allow concurrency per instance — a genuine shared-mutable-state hazard in code that looks single-request (Shared Mutable State in Concurrency & Parallelism).
- A frozen environment can resume mid-operation much later, so an in-flight outbound call may complete long after the request it belonged to.
- Each function is a separately-permissioned unit. That is an opportunity: scope its identity to exactly the resources that handler needs, rather than giving one broad role to a whole service (Least Privilege in Security Engineering).
- Environment variables are the default configuration channel and are visible to anyone who can read the function's configuration. Secrets belong in a secret manager, fetched at initialisation (Secrets Are Not Configuration).
- A reused execution environment carries module-scope state between invocations, potentially between different users' requests. Never cache per-user data at module scope.
- The dependency bundle is your supply chain, packaged and uploaded. Scan it like any other artifact (Dependency Security).
- "Serverless is cheaper." Sometimes, for some traffic shapes. It removes idle cost and adds per-request cost; a steadily busy service can easily cost more than the instances it replaced. Model your own traffic profile rather than accepting the claim.
- "Serverless scales infinitely." The function scales. Your database, your third-party APIs and your quotas do not, and the function's scaling is what exposes them.
- "No servers to manage means no operations." You now manage concurrency limits, cold starts, timeouts, retry semantics and connection strategy — a different operational surface, not an absent one.
- "It is just my handler in the cloud." The process may not survive the response. Anything you deferred, batched or cached across requests needs redesigning.
- "Cold starts are a solved problem." They are reducible — small packages, cheap initialisation, warm capacity — and warm capacity is the thing that removes scale-to-zero, which was the reason to be here.
Operating it
- Separate cold and warm invocations in your metrics — a flag set at module scope and cleared after the first invocation is enough. Mixing them makes latency data meaningless.
- Track concurrent executions against the platform limit and against the database connection budget. That comparison predicts the next incident.
- Alert on throttled invocations and on duration approaching the configured timeout, which is the early warning for truncation.
- Flush traces and logs before returning the response, or accept that the interesting failures will be the ones with no telemetry (Tracing From the Backend's Side).
- Scaling up is close to instantaneous and is often the problem: the platform will happily create concurrency your database, your third-party API and your rate limits cannot absorb (Unbounded Concurrency).
- At sustained high throughput the per-invocation model stops resembling the workload it was designed for, and a long-lived process serving many concurrent requests is usually the better fit.
- Cold starts matter most at low volume and at the edges of a scale-up event — the two situations where the least traffic and the most new environments coincide.
- Cost shape, not cost level. Serverless converts a fixed cost into a per-invocation one: near zero when idle, and rising with every request. Whether that is cheaper depends entirely on the traffic profile, the duration and memory of each invocation, and what you would otherwise have provisioned. It is not always cheaper, and at steady high throughput it frequently is not. The honest framing is that it removes the cost of idle capacity and adds a cost per unit of work.
- You give up process-lifetime tricks — in-process caches, connection pools, background timers, warm state — that a long-lived server gets for free.
- Operational simplicity is bought with platform coupling: the concurrency, retry and timeout semantics you build around are the platform's, and they differ elsewhere.
- Debugging is harder: no process to attach to, no local reproduction of the freeze semantics, and telemetry that can be cut off mid-flush.
Where this applies
Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.
- CLOUD-SPECIFICConcurrency per environment, maximum duration, freeze semantics, retry behaviour of each event source and the availability of a connection proxy all differ by provider and by product. A design that works on one function platform can fail on another with the same code (Mapping Services Across Cloud Providers).
- SCALE-SPECIFICStrongest at low, spiky or bursty volume where idle capacity would dominate. At sustained high throughput the per-invocation model works against you on both cost shape and connection pressure.
- RUNTIME-SPECIFICCold-start cost varies by runtime: interpreted runtimes with small packages start quickly, JIT-based runtimes pay warmup on top of start, and a compiled static binary starts fastest. The same handler logic has materially different tail latency depending on language.
Where the depth lives
This domain teaches the application-side mechanism and hands the rest off.