SerializationGENERALFRAMEWORK-SPECIFICLANGUAGE-SPECIFIC

Deserialization: Bytes to Objects

Parsing untrusted bytes produces data of a known shape, not data you may trust — and the gap between those two is where mass assignment lives.

What actually happensHow to build it

The requirement, the obvious build, and why it breaks

Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.

The question

What has to happen to untrusted bytes before they are allowed to become a domain object?

The requirement

A client sends a JSON body to update a user profile. The handler needs a well-formed, validated, authorised update — and the body arrived from the open internet.

The obvious build

Parse the body and apply it: Object.assign(user, req.body) then save. The client sends the fields it wants to change and the ORM writes them. It is concise, it works, and every framework tutorial shows something close to it.

Why it breaks

A caller adds "role": "admin" to the body. Nothing rejected it, because nothing enumerated what was allowed — the update wrote a column the endpoint was never about (Mass Assignment and Over-Posting in Security Engineering is the attacker's view of this exact bug).

How it breaks in production
  • A caller adds "role": "admin" to the body. Nothing rejected it, because nothing enumerated what was allowed — the update wrote a column the endpoint was never about (Mass Assignment and Over-Posting in Security Engineering is the attacker's view of this exact bug).
  • A field arrives as "12" instead of 12. A dynamically typed runtime stores the string, and the comparison quantity > 10 silently starts doing string comparison somewhere downstream.
  • A body arrives with 200 MB of array. The parser materialises all of it before any handler code runs, and one request takes the process out of memory (Memory Leaks in Backend Services covers the diagnosis side).
  • A deeply nested object — arrays inside arrays, thousands deep — costs the parser far more CPU than its byte count suggests, and on a single-threaded runtime that stalls every other in-flight request (Blocking the Event Loop).
  • A null arrives where an object was assumed. The handler dereferences it and returns a 500 for what is really a 400.
RequirementAPI ContractApplication LogicData AccessExternal DepsConcurrencyFailureSecurityObservabilityDeploymentScale

What is actually happening

  • Parsing and validating are different steps and they are routinely fused. A parser answers "is this well-formed in this format". It says nothing about whether the fields are the ones you accept, whether their values are legal, or whether this caller may set them (Parse, Do Not Validate).
  • The parser runs before your code. Size limits, depth limits and content-type checks that live in your handler run too late — the body was already read into memory and already parsed.
  • Type coercion happens at the boundary, or does not happen at all. JSON gives you six types; your domain wants dates, decimals, enums and identifiers. Something must convert, and if nothing does explicitly, the untyped value travels inward.
  • Assignment by iteration is an allowlist you did not write. Object.assign, **kwargs splatting, Model(**body) and ORM update(attrs) accept whatever keys the payload had. The set of writable fields is defined by the model, not by the endpoint.
  • Some formats deserialize into arbitrary types. Language-native formats — Python pickle, Java serialization, PHP unserialize, YAML tags — reconstruct objects and can invoke constructors during parsing. Feeding those untrusted input is remote code execution, not a validation bug (Security Engineering's *Unsafe Deserialization*, linked below, has the exploit mechanics).

The five steps between bytes and a domain object

The reason "we validate our inputs" is not a sufficient answer is that it names one of five steps. Each step has its own failure and its own place in the stack, and skipping any of them leaves a specific, well-known bug.

Inbound body, end to end
  1. 1
    Read bytes (bounded)

    Reads the body up to a configured maximum, or rejects with 413 before buffering it.

    fails by No limit, or a limit only on some routes: one request buffers until the process dies.

  2. 2
    Decode + dispatch by content type

    Interprets bytes as text in a charset and selects the parser named by Content-Type.

    fails by Sniffing the format from the body, so the caller chooses which parser runs.

  3. 3
    Parse (bounded)

    Turns text into a generic structure, with depth and key-count limits.

    fails by Deep nesting burning CPU; a parse error surfacing as 500 instead of 400.

  4. 4
    Schema-parse into an input type

    Names every accepted field and its type, converts, and rejects everything else.

    fails by Skipped entirely — this is where mass assignment happens.

  5. 5
    Construct domain values

    Produces Money, Email, UserId — types that cannot hold an unchecked value.

    fails by Passing raw maps inward, so every later layer must re-ask "was this checked?"

Authorization is deliberately not in this list. It is a separate decision, made after you know what is being asked, and it belongs to Authorization in Backends.

The one-line version and the version that survives review

The compact form is not popular because engineers are careless. It is popular because it is genuinely shorter, it adapts automatically when the model gains a field, and in a codebase with one trusted client it does the right thing for a long time.

That automatic adaptation is exactly the defect. The set of fields a caller can write grows every time anyone adds a column, and no review of the migration ever mentions the endpoint.

Updating a profile
Assign the parsed body
app.patch('/me', async (req, res) => {
  const user = await repo.byId(req.user.id)
  Object.assign(user, req.body)   // whatever arrived
  await repo.save(user)
  res.json(user)                  // and whatever the row holds
})
Parse into an endpoint input type
const ProfileUpdate = z.object({
  displayName: z.string().min(1).max(80).optional(),
  locale: z.enum(['en', 'de', 'fr']).optional(),
}).strict()                       // unknown keys rejected

app.patch('/me', async (req, res) => {
  const input = ProfileUpdate.parse(req.body)
  const user = await repo.byId(req.user.id)
  const updated = await repo.updateProfile(user.id, input)
  res.json(toProfileResponse(updated))
})

The writable surface of the endpoint is now a list you can read, and it does not change when someone adds a role, credit_balance or is_internal column. The response is a mapped type for the same reason (Schema Leakage).

What each shortcut actually produces

Every row here starts as a reasonable simplification. The middle two columns are what an on-call engineer sees; the last column is the fix that removes the class rather than the instance.

TriggerSymptomCauseResponse
Body assigned onto the entityA user is suddenly an admin; an order has a price nobody setThe model, not the endpoint, defined the writable fieldsAn endpoint input type with unknown keys rejected
No body-size limit on one routeOne instance OOMs under a single request; restart loopThe parser buffers before any handler code runsLimit at the edge and in the framework, per route, from measured payload sizes
Deeply nested payloadp99 spikes across every endpoint on the instanceParser CPU is superlinear in structure, not linear in bytesDepth and key-count limits; on a single-threaded runtime this is a liveness control (Blocking the Event Loop)
Coercion left to the runtimeA comparison starts behaving lexicographically in production only"12" was never converted to a numberConvert at the boundary into a domain type that cannot hold a string
Native-format deserializer on user inputArbitrary code executes as the service accountThe format reconstructs objects, not dataData-only formats for anything from outside the process; Security Engineering has the exploit side

How to build it

Most important first.

  • Bound the input before parsing: maximum body size, maximum nesting depth, required Content-Type. These belong at the edge and in the framework configuration, not in the handler (Request Bodies and Streaming, Resource Limits).
  • Parse into an explicit, endpoint-specific input type — a schema, a struct, a dataclass — that names every field you accept and its type. Fields that are not named do not exist as far as the handler is concerned.
  • Decide what an unknown field means and apply it uniformly. Rejecting them catches client typos early; ignoring them makes rolling deployments and old clients easier. Both are defensible; silently persisting them is not.
  • Convert to domain types at the boundary — a Money, a UserId, an Instant — so that no code inward of the boundary has to ask whether a value was ever checked (Parse, Do Not Validate).
  • Validate in the three separate places that exist: shape at transport, rules in the business layer, invariants in the database (The Three Validations).
  • Never deserialize untrusted input with a language-native format. If you must accept one, use a format that constructs only data.

What can go wrong

Failure modes
  • The size limit is set on one route and not another, and the unlimited one is the file upload nobody reviewed.
  • A schema library is used for shape only, and the handler then reads body.extra directly — bypassing the schema for exactly the field the schema was protecting.
  • Coercion is too permissive: a validator that accepts "true", "1" and "yes" as booleans also accepts "no" as truthy in some libraries.
  • A parse failure returns 500 instead of 400, so a client bug pages you and shows up in your error budget (An Error Taxonomy That Maps Cause to Response).
  • Validation errors expose the internal field names of your model, which is a small schema leak and a helpful map for an attacker (Reporting Validation Failures).
What can race
  • A partial update parsed from a body has no idea what the record looked like when the client rendered its form. Two concurrent PATCHes to different fields of the same row can each write a full row and lose the other's change (Optimistic Concurrency).
Security
  • Mass assignment is the headline: without an explicit allowlist, the set of writable fields is your database schema, and the endpoint's intent is irrelevant.
  • Authorization is a separate step from validation and comes after it. A perfectly valid update to someone else's record is still an attack (Object-Level Authorization).
  • Parser resource consumption is a denial-of-service surface: size, depth, key count, and in some formats duplicate keys and enormous numbers.
  • Content-type confusion — accepting form encoding on a JSON endpoint, or sniffing the format from the body — changes which parser runs and can defeat CSRF assumptions (Cross-Site Request Forgery (CSRF) in Security Engineering).
  • Never eval, never a native deserializer, never a YAML loader with tag construction enabled on input from outside the process.
Misreads
  • "The schema library validated it, so it is safe." It validated shape and values. It has no idea who is calling or which record they may touch.
  • "Our client only sends the right fields." Your client is one client. curl is another, and so is the mobile app from two releases ago.
  • "It parsed, so the types are right." It parsed, so the JSON was well-formed. In a dynamic runtime, "12" parses beautifully.
  • "Unsafe deserialization is a Java problem." It is a property of formats that reconstruct arbitrary types, and every major language has at least one.

Operating it

How you see it in production
  • Count 400s by route and by failing field. A spike on one field after a deploy is a client that broke, and you will know before they file it.
  • A histogram of request body size per route, so the day someone starts posting 50 MB you find out from a graph rather than from an OOM.
  • Log the parse-versus-validate distinction: "malformed body" and "invalid value" are different alerts with different owners.
  • Track rejected unknown fields if you reject them — a rising count usually means a client is ahead of your deploy.
What changes at 10x and 100x
  • Parsing is CPU proportional to body size and structure. At 10x request rate it is 10x CPU with nothing to amortise it against.
  • At 100x, a body-size limit becomes a capacity decision rather than a safety one: the limit times the concurrency ceiling is memory you must actually have.
  • Large inputs eventually have to be streamed rather than buffered, which means validating incrementally and rejecting after you have already accepted bytes.
What this costs
  • Explicit input types cost boilerplate per endpoint and one more place to forget a new field. In exchange the writable surface of every endpoint is visible in one file.
  • Rejecting unknown fields catches client mistakes and makes rolling deploys harder — during a rollout, a new client can be talking to an old server that has never heard of the field.
  • Tight limits protect the process and reject the legitimate large request. The limit must come from measured real payloads, not from a round number.

Where this applies

Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.

  • GENERALThe parse/validate/authorize separation applies to any input from outside the process, over any protocol.
  • FRAMEWORK-SPECIFICMass-assignment protection differs sharply: Rails and Laravel ship allowlist mechanisms (strong parameters, $fillable) that are opt-in and easy to widen; Express, FastAPI and most Go frameworks provide none by default and rely on you to define an input type. Knowing which one you are in decides whether "the framework protects me" is true.
  • LANGUAGE-SPECIFICStatically typed decoders (Go, Rust, Java) reject unknown or ill-typed fields at the decode step by default or by one flag; dynamically typed ones accept whatever arrived and defer the discovery to runtime, often several layers inward.

Where the depth lives

This domain teaches the application-side mechanism and hands the rest off.