What Serialization Costs
CPU proportional to the nodes you visit and garbage proportional to the bytes you produce — paid on every request, amortised by nothing.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.
Why does an endpoint whose only query is fast still burn CPU and allocate heavily?
The list endpoint returns 200 items per page. The query is indexed and quick, and the endpoint is still the most expensive thing the service does.
The database is the slow part of a backend. If the query is fast, the endpoint is fast, and anything left over is framework overhead that is not worth investigating.
The trace shows a fast query and a long gap afterwards with nothing in it. The gap is encoding, and nothing instruments it by default.
- The trace shows a fast query and a long gap afterwards with nothing in it. The gap is encoding, and nothing instruments it by default.
- CPU rises with traffic while the database stays bored, and autoscaling adds instances to do more encoding rather than fixing what is encoded (Autoscaling a Backend).
- Memory sawtooths hard: each response allocates an intermediate structure, a string and a byte buffer, all of them garbage moments later. Pause times rise with load rather than with data size.
- On a single-threaded runtime, encoding a large payload occupies the loop, so an unrelated fast endpoint on the same instance sees its latency track the size of somebody else's response (Blocking the Event Loop).
- Enabling compression to shrink the payload moves the cost rather than removing it: fewer bytes, more CPU per byte, on the same saturated core.
What is actually happening
- Cost tracks nodes visited, not fields you meant to send. A 200-item page where each item has 30 fields and one nested object is on the order of ten thousand property visits per response. That number is arithmetic, not a benchmark, and it is the number to reason with.
- Producing text allocates. The encoder builds intermediate values, grows buffers, and emits a string that is then encoded to bytes. Most of it is short-lived garbage, which is cheap per object and expensive in aggregate (Allocation Rate Is a Cost Even Without a Leak in Observability & Performance).
- Escaping scans every character. Strings must be checked for characters needing escapes, so string-heavy payloads cost in proportion to their total text length, independent of field count.
- Your code runs inside the walk. Getters, computed properties,
toJSONhooks and lazy relations execute per node. A property that looks free in the model is a function call per item, and occasionally a query per item (The N+1 Query Problem). - Nothing amortises it. Unlike a query, an encode has no cache, no plan reuse and no index. It is paid in full on every request unless you cache the produced bytes.
- Where it hurts depends on the runtime. The same CPU cost is tail latency for unrelated requests on a single-threaded event loop and plain CPU saturation on a thread-per-request runtime.
Where the time and the garbage actually go
Encoding is not one cost, it is four, and they respond to different fixes. Knowing which one you are paying is the difference between deleting fields and replacing a library.
The arithmetic is worth doing by hand once. A page of 200 items with 30 fields each visits roughly six thousand properties before any nested object is counted. Halving the page and dropping ten fields is a four-fold reduction in visits, and it costs nothing but a conversation with the client team.
- Graph traversal — one visit per property, plus a function call for every getter, computed field and
toJSON. Fix by visiting fewer nodes. - Text production — buffer growth and intermediate strings, most of it immediate garbage. Fix by producing fewer bytes.
- Escaping and encoding — proportional to total character count, then again for UTF-8. Fix by sending less text.
- Compression, if enabled — additional CPU per byte in exchange for fewer bytes on the wire. Fix by moving it to the proxy, or by not needing it.
1// 200 rows x every column the table has, then a walk over all of it2const orders = await db.query('select * from orders limit 200')3return res.json(orders)4 5// 200 rows x the six fields the client reads6const orders = await db.query(7 'select id, status, total_minor, currency, created_at, customer_id\n from orders limit 200'8)9return res.json(orders.map(toOrderSummary))Both versions run one indexed query. The difference is how many values are fetched, constructed as runtime objects, walked, escaped and written — and the same change also fixes the contract problem in Schema Leakage.
Send less, then encode faster
The ordering in this section is not stylistic. Reducing what you send improves query time, object construction, encode time, bandwidth and client parse time together. Changing the encoder improves one of those and costs you tooling.
This is also why "switch to Protobuf" so often disappoints: the payload was large because it contained the whole row, and it still contains the whole row afterwards, in fewer bytes.
// keep returning full entities, 500 per page // swap the JSON encoder for a faster one // -> encode time falls; query time, object // construction, bandwidth and client parse // time are unchanged
// project the six columns the client reads // cap the page at a size the client actually uses // drop the nested expansion nobody renders // -> fewer rows fetched, fewer objects built, // fewer nodes walked, fewer bytes shipped, // less to parse on the client
Encoder speed is one term in the cost. Node count is a term in every stage of the request, from the query planner to the client's parser, so it is where the leverage is.
Which lever, and what it costs you
Once a profile actually blames encoding, these are the moves, roughly in order of how much they buy for what they cost. There is no default answer: a personalised feed and a public price list have different correct choices.
The profile says encoding is hot. What do you change?
when The response contains columns no client reads — the common case for SELECT * endpoints.
cost A contract conversation, and possibly a version, if a client is reading one of them (Removing Fields Without Removing Consumers in API Design).
when Clients render a screenful and receive hundreds of rows.
cost More round trips; worse over a high-latency link.
when Many callers get a byte-identical response — public catalogues, config, reference data.
cost Invalidation, and a cache key that must include every dimension of variation including identity.
when Genuinely large collections consumed incrementally.
cost No clean error after the first byte; backpressure becomes your problem (Backpressure).
when Reports and exports.
cost A job, a status endpoint and a storage lifecycle (The Async Job Pattern in API Design).
when A profile blames the encoder specifically, and consumers can adopt a schema toolchain.
cost Debuggability, a build step, schema evolution discipline — and it does nothing about node count.
How to build it
Most important first.
- Send less before encoding faster. Fewer fields and smaller pages reduce query cost, encode cost, bandwidth and client parse time at once; a faster encoder reduces one of the four (Pagination That Survives a Large Table, Over-Fetching and Under-Fetching in API Design).
- Select only the columns you serialize. Fetching a wide row to emit six fields pays for the fetch, the object construction and the walk over everything you discarded.
- Profile before changing format. The encoder shows up in a CPU profile by name; if it is not near the top of the flame graph, replacing it buys nothing (Reading a Flame Graph).
- Cache the produced bytes, not the objects, when a response is identical for many callers — and be explicit that you now own invalidation of a serialized artefact (Cache Invalidation).
- Stream large collections rather than building one buffer, accepting that errors after the first byte can no longer be a status code (Request Bodies and Streaming).
- Let the proxy or CDN do compression when you have one, so the CPU is spent somewhere with more of it (Compression: Cheaper Bytes, Not Fewer in API Design covers the contract side).
- Keep response objects flat and free of backreferences — a response type is not an object graph (Three Models, Not One).
What can go wrong
- A hidden lazy relation makes encoding cost look like database cost, or the reverse, depending on where your instrumentation sits.
- Caching serialized bytes for a personalised response, so one user receives another user's body — a cache-key bug with a security outcome.
- Streaming implemented without backpressure: the encoder produces faster than the socket drains and the buffer becomes the memory leak (Backpressure).
- Switching to a binary format for a browser client, where the client-side decode library costs more than the bytes saved.
- Compression enabled on an already CPU-saturated instance, turning a bandwidth problem into a latency problem.
- A serialization cache holds bytes produced from a snapshot. A concurrent write updates the row and invalidates the entry, and a request that started before the invalidation can still write the stale bytes back into the cache (Cache Invalidation).
- Encoding cost is attacker-reachable. If a caller can request
?limit=100000or a deeply expanded resource, they choose how much CPU you spend — bound page size and expansion depth server-side (Resource Limits). - Compressing a response that mixes a secret with attacker-controlled content can leak the secret through compressed size; this is the family of attacks behind disabling compression on some sensitive responses (TLS as a Security Boundary in Security Engineering).
- A cached serialized response must be keyed by everything that varies it, including the caller's identity and permissions. Serialization caches are a common source of cross-user disclosure.
- "JSON is slow, we should move to Protobuf." Sometimes true, usually premature. A profile decides it; on most services the payload size and the object count are the cost, and both survive the format change.
- "Compression makes responses cheaper." It makes them fewer bytes and more CPU. Whether that is cheaper depends on which resource you are short of.
- "The query is fast so the endpoint is fast." Only if you measured the part after the query.
- "Serialization cost is framework overhead." It is proportional to your data shape, and you control the data shape.
Operating it
- A CPU profile under real traffic. Encoder frames appear with their own names; this is the one measurement that settles the argument (Self Time, Total Time, and Where the CPU Went).
- An allocation profile, because encoding often shows up as GC pressure before it shows up as CPU time.
- A span around the encode step in the request trace, separate from the handler and the query.
- Response bytes per route as a histogram, plotted next to request rate. Payload growth is the usual root cause and the easiest to miss.
- On an event-loop runtime, loop lag correlated with response size — that correlation is the proof that encoding is the blocker.
- At 10x requests, encode cost is 10x. There is no cache, no index and no connection pool that changes that; only fewer nodes or cached bytes do.
- At 100x, bandwidth and client parse time typically outweigh server encode time, which shifts the win from "a faster encoder" to "a smaller response".
- Large exports stop being request-path work at all and become a job that writes to object storage and hands back a link (Background Jobs, Object Storage).
- Caching serialized bytes buys the most and costs invalidation plus a real risk of serving one caller's body to another.
- A binary format trims bytes and parse time and costs you readable payloads, a build step, and a schema you must now evolve carefully.
- Streaming removes the big buffer and removes your ability to fail cleanly after the first byte.
- Smaller pages reduce per-response cost and increase the number of round trips, which can be worse over a high-latency mobile link.
Where this applies
Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.
- GENERALThe node-count model and "nothing amortises an encode" hold everywhere.
- RUNTIME-SPECIFICNode runs the encode on the one loop thread, so a large response raises latency for every unrelated request on that instance; Go and the JVM spread encodes across threads, so the same work appears as CPU saturation and a throughput ceiling rather than as cross-request latency. CPython's GIL sits between the two — a C-implemented encoder can release it, a pure-Python one cannot.
- SIMPLIFIEDTreating cost as proportional to node count ignores string length, escaping and allocator behaviour. It is the right first model for deciding what to change, and the wrong model for predicting a number — measure that.
Where the depth lives
This domain teaches the application-side mechanism and hands the rest off.