Streaming APIs: Partial Data as a Contract

Transport-level streaming (chunked transfer, HTTP/2 data frames) says only "bytes will keep arriving". The contract on top must define the unit — a token, a complete JSON object per line, a row, a typed event — and, more importantly, the prefix semantics: what is a consumer entitled to do with the first N chunks before the stream ends? For a token stream, the prefix is a growing draft the UI renders and the user reads — provisional by nature. For an export, the prefix is N durable rows the consumer may have already written to disk. For a progress stream, each chunk supersedes the last. Three streams, three different answers to "can I act on what I have?", and consumers guess wrong unless told.

The sharpest edge is failure after a valid prefix. A query that streams 80k rows and then hits an error has already delivered 80k perfectly-formed rows — the consumer's file is four-fifths of a lie unless the stream *ends with a verdict*. This is why streamed responses need an in-band termination signal: a final event: done with a summary (count, checksum), or a terminal error frame with a machine-readable code (The Error Model: Structure Over Apology still applies inside streams — arguably more, since the HTTP status was sent long ago and said 200). A stream that just stops is indistinguishable from a stream that was cut, and "the connection closed" is not a verdict.

Streams outlive the moment either side wants them. Client-initiated cancellation must be cheap and honored: the user closed the tab three tokens in, the exporter found what it needed after 2% of the data. The transport gives you the mechanics (connection close, HTTP/2 RST_STREAM, gRPC cancellation) — the contract's job is the semantics: does cancelling the stream cancel the *work*? For a pure read, yes, trivially. For a generation billed per token or a job producing side effects, the docs must say what a cancel stops, what it bills, and what state remains (Long-Running Operations: 202 and the Job Resource for when the work should be a first-class resource instead).

Server-side flow is the mirror problem: a consumer that reads at 1MB/s from a stream produced at 50MB/s forces someone to hold the difference. Transport backpressure (TCP flow control, HTTP/2 window) will push back into your producer — a good default, but the contract should state the consequence: slow consumers get a slower stream (producer pauses), or bounded buffering then disconnect with a too_slow error, or spilled-to-disk delivery. Pretending the question doesn't exist means unbounded buffers and the memory incident that follows (Slow Clients and Backpressure treats this in depth).

Resumption completes the lifecycle. Enumerative streams should be resumable by position — an opaque cursor or sequence per chunk, so a consumer that died at row 812,004 continues from there rather than re-downloading 10GB; this is Cursor Pagination: An Opaque Bookmark, Not a Position wearing a different coat, and the same rules apply (opaque tokens, stated validity window). Generative streams usually declare non-resumability honestly — regenerate instead — because replaying a half-finished generation is rarely meaningful.

Streaming earns its complexity from one of three pressures: the result is too large to materialize (exports, bulk reads — streaming bounds memory on both sides), the result is produced over time and early parts have value now (tokens, logs, progress), or the consumer needs lower time-to-first-byte than materialize-then-send allows. Absent all three, a plain response or a paginated sequence of plain responses (Pagination: Choosing How Lists End) is strictly simpler: cacheable, retryable, debuggable with curl, and immune to every mid-stream ambiguity this lesson exists to resolve.

When you do stream, shape the format for incremental consumption: newline-delimited JSON (or typed SSE events, or length-prefixed frames) — *not* one giant JSON array, which most parsers can only validate at the closing bracket, quietly re-materializing everything streaming was meant to avoid. And keep a non-streaming sibling where consumers plausibly want one (?stream=false, or the job-then-download pattern): scripts and batch integrations often prefer one complete, verifiable response over incremental delivery.