AgenticGENERALFRAMEWORK-SPECIFICDEVICE-SPECIFIC

Streaming a Response Without Melting the Device

Tokens arrive dozens of times a second on a thread that also has to paint. What you do per chunk decides whether the answer feels alive or the page feels broken.

What actually happens What it costs the browser How to build it

The intent, the obvious build, and why it breaks

Every lesson starts where the work starts: someone wanted an outcome, and the first implementation that comes to mind has a problem.

The question

How do I render an answer as it arrives without re-rendering the world on every token?

The user intent

Someone asked a question. They want to start reading the answer immediately rather than watching a spinner for thirty seconds, and they want to be able to scroll back while it is still coming.

The obvious build

Append each chunk to a state string and let the framework re-render. It is one line and it works in the demo.

Why it breaks

Each token triggers a render of the entire transcript, so the cost per token grows with the length of the conversation — the answer gets slower as it gets longer.

How it breaks in a real browser

Each token triggers a render of the entire transcript, so the cost per token grows with the length of the conversation — the answer gets slower as it gets longer.
Markdown is re-parsed from scratch on every chunk, turning a cheap append into a full document reparse dozens of times a second.
On a mid-range phone the main thread never gets a clear frame, so scrolling stutters for the entire duration of the response (Long Tasks).
Auto-scrolling to the bottom on every chunk fights the user the moment they scroll up to re-read something.
If the stream stops mid-sentence, the UI has no way to say so, and half an answer is presented as a whole one.

Intent→Event→State→UI Logic→DOM Work→Network→Layout / Paint→Pixels→Feedback

What is actually happening

In the browser, not in the framework.

A streamed response is delivered either as Server-Sent Events or as a readable body you consume chunk by chunk. Chunks are not token-aligned or message-aligned — a chunk can split a multi-byte character or half a JSON event, so buffering and framing are yours to handle (Server-Sent Events).
Each state update is a task on the main thread that can invalidate style, layout and paint for a growing subtree (The Cost of a Change).
Appending text to the end of a block is one of the cheapest DOM mutations available — the expensive part is almost never the append, it is everything the framework re-runs around it (What a Component Costs to Render).
Browsers coalesce paints to the display refresh, so producing more state updates than frames does strictly wasted work: several updates land in one frame and only the last one is ever seen (The Rendering Opportunity).

What this makes the browser do

And which of it is avoidable.

Decoding and buffering stream chunks, then splitting them into events.
Re-running whatever the framework re-runs per update — which is the whole cost, and the whole opportunity.
Markdown parsing and DOM construction, once per update unless you make it once per boundary.
Style and layout over an ever-growing transcript, unless it is windowed or contained (CSS Containment).

Flush on a frame, not on a chunk

The fix is small and the reasoning is worth internalising because it generalises far beyond agent UIs: a state update that happens more often than the browser can paint is partly wasted by construction. The display refreshes at a fixed cadence; several updates landing between two frames all collapse into whichever one was last (The Rendering Opportunity).

So buffer incoming text and flush once per animation frame. The user cannot perceive the difference, and you have converted an unbounded number of renders into at most one per frame.

What re-renders

Whole transcript is reactive

// every token re-runs the transcript
setMessages(msgs => [
  ...msgs.slice(0, -1),
  { ...last, text: last.text + token },
])

Only the live message is reactive

// finished messages are immutable and never re-render
<Transcript messages={finished} />
<StreamingMessage stream={stream} />
//   ^ owns its own buffered state; nothing above it updates

Everything above the cursor is already final. Encoding that in the component structure means the cost of a token stops growing with the length of the conversation — which is the difference between an answer that stays smooth and one that degrades as it goes.

Buffered flush

1let buffer = ''
2let queued = false
3 
4function onChunk(text: string) {
5  buffer += text
6  if (queued) return
7  queued = true
8  requestAnimationFrame(() => {
9    queued = false
10    const pending = buffer
11    buffer = ''
12    appendToStreamingMessage(pending)   // one update, whatever arrived
13  })
14}

The whole idea is the queued guard: many chunks, at most one update per frame. Nothing else in this lesson matters as much.

Scrolling that does not fight the reader

Following the bottom of a growing answer is right up until the moment the user scrolls up, at which point continuing to scroll is actively hostile: they are trying to re-read something and the page keeps yanking them away.

The rule is to follow only while the user is already at the bottom. Detect that they have left, stop following, and offer an explicit way back — which also gives keyboard and screen-reader users a control they can actually use, instead of behaviour that only works if you are watching.

Streaming UI failures and what causes them

Trigger	Symptom	Cause	Response
User scrolls up mid-answer	Page snaps back to the bottom repeatedly	Unconditional scroll-to-bottom on every chunk	Follow only while already at the bottom; offer a "jump to latest" control.
Long conversation	Typing in the prompt box lags	Every token re-renders a transcript of thousands of nodes	Isolate the live message; window the transcript (List Virtualization).
Connection drops mid-stream	Half an answer, presented as finished	No terminal state distinguishing complete from truncated	Model termination explicitly and render `truncated` differently, with a way to continue.
Chunk splits a character	A replacement glyph appears in the text	Decoding each chunk independently	Use a streaming decoder that carries state across chunks.
Screen reader in use	Continuous stuttering announcement; user cannot read the answer	Streaming text into an `aria-live` region	Announce state changes only; leave the answer as ordinary readable content (Live Regions and Announcement).
Reconnect after a drop	Part of the answer appears twice	Resuming without an offset or last-event id	Resume from a cursor and discard what has already been rendered (Resynchronisation After a Gap).

What the user is actually waiting for

It is worth being precise about why streaming helps, because it explains what to optimise. The total time to a complete answer is essentially unchanged by streaming. What changes is the time until the user can begin reading, and that is the number that determines whether the wait feels like progress or like a hang.

The timeline below is schematic. What transfers is the shape: the gap before first token is the only part the user experiences as waiting, and the rest is experienced as reading.

Waiting versus readingrelative units — a shape, not a measurement

Request sent

Server + model latency

First token

Streaming (user reading)

Tool call mid-run

Complete

Server + model latency — The only genuine wait. Everything the user perceives as "is it broken?" happens here, which is why an immediate, announced pending state matters more than anything else in this lesson.
First token — Perceived responsiveness is decided here, not at completion.
Streaming (user reading) — Experienced as reading rather than waiting — provided the main thread stays free enough to scroll.
Tool call mid-run — A second wait, in the middle. Unannounced, it reads as a freeze (Showing What the System Is Doing).

Two waits, not one. The mid-run tool pause is the one teams forget, and it is the one that looks most like a bug.

How to build it

Most important first.

Buffer and flush on a frame rather than on a chunk. Batching to roughly one update per animation frame gives the user the same perceived immediacy at a fraction of the work — nothing that renders more often than the display refreshes is visible anyway (Yielding and Scheduling).
Isolate the streaming block. Only the message currently being written should re-render; everything above it is finished and immutable (Node Identity Across Updates).
Parse markdown incrementally, or render the streaming message as plain text and convert it once at the boundary. A full reparse per chunk is the single biggest avoidable cost here.
Window a long transcript rather than keeping thousands of nodes alive (List Virtualization).
Make auto-scroll conditional: follow the bottom only while the user is already at the bottom, and stop the instant they scroll away (Scroll Restoration).
Give the stream a real terminal state, including truncated, and render it differently from complete.

Keyboard, focus, semantics, announcement

A required field on every lesson in this domain, not a section added when there is room.

Do not put the streaming text in an aria-live region. A live region announces changes as they happen, so token-level streaming produces a token-level announcement stream: an unusable stutter that cannot be paused, skimmed or re-read (Live Regions and Announcement).
Announce state, not content: "Generating response" when it starts and "Response complete" when it ends, in a polite live region, while the answer itself lives in ordinary readable content the user navigates on their own terms.
Keep the answer reachable as static text as soon as it exists. A screen-reader user should be able to read the finished part with normal reading commands while the rest is still arriving.
Do not move focus when the response completes — that interrupts whatever the person was doing. Announce completion instead, and let them choose to navigate to it (Focus Management).
Respect prefers-reduced-motion for typing effects, caret animations and shimmer placeholders; a simulated typewriter is decoration and some people find it actively difficult (Contrast, Colour and Motion).

What can go wrong

Failure modes

A chunk splits a multi-byte character and the UI renders a replacement character, or splits a JSON event and the parse throws mid-stream.
Auto-scroll fights the user, so re-reading an earlier paragraph becomes impossible while the answer is still arriving.
The transcript keeps every message and every tool result forever, so memory grows for as long as the tab is open (Memory Leaks).
Reconnection restarts the answer from the beginning, duplicating the text the user already read.
A network drop is rendered as a finished answer, which is worse than an error because it is not detectable.

What can arrive out of order

A cancel can arrive between two chunks, so the UI must decide what the already-rendered partial text means — and say so.
A reconnect can deliver text the client already rendered; without an offset or event id, the answer duplicates (Ordering and Duplicate Delivery).
A second prompt submitted mid-stream produces two live streams unless the first is cancelled or they are keyed independently (Out-of-Order Responses).

Security

Do not render partial markdown as HTML while it is still arriving. A half-received construct can produce markup you did not intend, and a sanitizer that runs on incomplete input is easy to slip past (Sanitization and Trusted HTML).
Render the streaming text as plain text, and convert to rich content once at a completed boundary, through a sanitizer you control (Cross-Site Scripting).
Links and images in model output point at attacker-influenceable URLs. An image loaded from model output leaks a request to whoever controls that host (Third-Party Scripts and the Supply Chain).

Misreads

"Streaming makes it faster." It makes it *feel* faster by starting sooner. Total time is unchanged, and a bad implementation makes the device slower throughout.
"More updates means smoother." Beyond one per frame, extra updates are invisible and cost real work.
"The framework will batch it." Some batch some updates; none of them know that your transcript above the cursor is immutable. That is your structural information to exploit.
"A typing animation makes it feel more natural." It also delays the content, and for some people it is a barrier rather than a flourish.

Measuring it, and what changes in the field

How you would see this

Time to first token, separately from total duration — it is the number that decides whether the wait feels responsive (Interaction Responsiveness).
Main-thread time attributed to the streaming component in a Performance recording. The re-render-per-token mistake is unmistakable there: a solid wall of scripting with no idle gaps (Debugging Rendering and Jank).
Updates per second versus frames per second. If the first is much larger than the second, that difference is pure waste.
Dropped frames during streaming, which is what the user actually feels as jank (The Frame Budget).

Slow device, slow network, large data, old tab

On a slow device the per-update cost is multiplied across the whole response, so a design that is fine on a laptop can be unusable on a phone for the entire minute the answer takes.
On a slow network, chunks arrive in fewer, larger pieces — which is easier on the main thread but makes the answer feel lumpier.
In a long conversation, everything scales with transcript length unless the transcript is windowed.

What this costs

Batching to a frame adds a tiny amount of latency per token and removes most of the rendering cost. The latency is imperceptible; the cost is not.
Rendering plain text while streaming and converting at the end is safer and cheaper, and costs a visible reflow at the boundary — worth designing for rather than avoiding.
Windowing the transcript bounds cost and complicates find-in-page and scroll restoration (List Virtualization).

Where this applies

Frontend advice ages badly and fragments across engines. These labels say what each claim is specific to, and where a different browser, device or framework would differ.

GENERALBatching to the frame, isolating the mutable block and not streaming into a live region follow from the browser's rendering and accessibility models, so they hold across frameworks and vendors.
FRAMEWORK-SPECIFICHow much a per-chunk update costs depends on the reactivity model: React re-runs the component and diffs, so an unisolated transcript is expensive; Solid and Svelte update the specific text binding, so the same naive code costs far less. The isolation advice matters most in the first case and is still worth doing in the others (Reactivity Models).
DEVICE-SPECIFICOn a fast desktop a per-token re-render of a short transcript is genuinely unnoticeable, which is why this defect ships; on a mid-range phone with a long conversation the same code drops frames for the whole response.

Where the depth lives

This domain teaches the browser-side mechanism and hands the rest off.

API Designstreaming-apis

Securityxss

Performancellm-latency

Agentictoken-budgets

Domains that do not exist yet

— Programming Languages & Runtime Internals — incremental parsing of a growing document, and why reparsing from the start is quadratic in the length of the answer.