How real-time LLM streaming works with SSE and browser streams
Follow an LLM response through a secure server, typed provider events, SSE parsing, browser streams, rendering, cancellation, and backpressure.
The whole story in 6 lines
A reliable LLM stream preserves boundaries, order, ownership, flow signals, and terminal state from provider to pixels.
- A trusted application server keeps provider credentials away from the browser while relaying the stream.
- Providers emit typed lifecycle events, and only delta events contribute new text to the answer.
- SSE boundaries come from blank lines, not from the arbitrary chunks returned by a network reader.
- Browser code decodes bytes, preserves partial text, parses complete frames, and dispatches typed events.
- A single message buffer preserves order while scheduled paint batches keep rendering work bounded.
- Pressure is local until each layer propagates it, while cancellation requires an explicit upstream signal.
Setup
A chat answer begins appearing, then freezes halfway through a word. The server is still busy, so we need to find which boundary lost the rest.
A delta is only the new text added by one provider event. It is a fragment, not necessarily a token, word, or complete sentence.
Server-Sent Events, called SSE, wrap messages in UTF-8 text fields. A chunk is merely one transport piece returned by a stream reader.
Backpressure is a signal that a consumer needs slower delivery. Our map now separates request, provider events, framing, browser parsing, painting, and shutdown behavior.
These four terms give us handles for the failure without pretending every boundary behaves alike. Now let us start with the trusted request path.
Request boundary
The browser begins with a prompt and a fresh AbortController. It can stop its own request later, but it should not contain a secret provider credential. That separation gives the user control without granting provider authority.
The prompt moves to the application server over the application’s authenticated route. This server can check the user, apply limits, and record request ownership. It also gives the request one server-side owner.
The provider now needs an API credential, while anyone can inspect downloaded browser JavaScript. Secrets are different from ordinary request data. Which side of this boundary should own the secret key?
The locked key stays inside the trusted server while an authorized request moves toward the provider with streaming enabled. A response lane opens immediately for later events. Nothing secret travels back with the response.
This boundary separates public interface code from provider authority. The browser only receives application-owned stream data. Next, we will inspect what travels back through the newly opened response lane.
Provider events
Now that the secure request is open, the provider begins a typed event sequence. The first created event establishes response state without adding answer text. Think of this as opening an ordered event ledger.
A text delta event carries only the next fragment, which our illustrative response begins with “Pigments.” The application keeps the event type beside that payload. That fragment can end anywhere inside a word.
Another provider record arrives while the answer is still incomplete. Lifecycle records still matter, but they do not extend the message. Which event type should extend the text instead of changing only lifecycle state?
The second delta joins the first fragment in order, so one answer now reads “Pigments shift.” Created and completed remain separate lifecycle records around it. The order matches the provider’s event sequence.
Typed events let code handle content, completion, and failure without guessing from text. A failure event could end the ledger differently. Next, those events need boundaries that survive real network delivery.
SSE framing
★ If you remember one thing · Network chunks are transport pieces, so only buffered SSE framing can recover complete provider events.
We just identified the provider’s typed records. SSE serializes each record as UTF-8 text fields, then a blank line marks the message boundary. The event field names the record type.
The network cuts that continuous text into delivery chunks based on transport timing and buffering. A cut can land inside event, data, or JSON text. Those cuts do not change the underlying SSE text.
A simple loop receives one chunk and immediately tries to parse it as one event. The shortcut confuses transport timing with message grammar. Will every valid stream survive that shortcut?
The same chunks now take two paths. Per-chunk parsing leaves broken fragments, but the buffered path joins leftovers and recovers every complete event. Both paths received identical bytes in identical order.
Here is the key: preserve the remainder, extract complete blank-line frames, and keep the next partial frame. This small buffer is the parser’s essential memory. Next, the browser will run that pipeline.
Browser pipeline
Now that blank lines define event boundaries, browser code starts from the fetch response body. Its reader returns byte chunks plus a done flag over time. The body remains open while later bytes arrive.
A streaming text decoder converts those bytes to characters while preserving an incomplete UTF-8 character between reads. That prevents a split character from becoming corruption. This matters for characters encoded across several bytes.
Decoded text joins the same remainder buffer from our framing lesson. Only complete blank-line frames move forward, while unfinished text waits for another read. The buffer can therefore span several reader results.
One complete frame becomes a typed JavaScript object, and its delta field enters the content handler. The handler never receives half a JSON payload. Typed dispatch can now branch on the event name.
The browser pipeline changes representation at each step without changing order. Each conversion hands complete material to the next layer. Next, we will append those ordered deltas while controlling how often the page paints.
Incremental render
The browser handler now owns ordered text deltas. It starts one assistant message buffer and appends each fragment instead of creating a new message per event. That buffer is the partial answer’s source of truth.
The first delta extends the buffer, and the interface can show useful text before completion. The partial answer remains marked as streaming rather than final. Completion will later change status without rewriting history.
Several small deltas can arrive between browser paints. The scheduler can wait briefly without reordering data. Should the interface replace the whole conversation or patch the one growing message?
Two pending fragments merge into one scheduled patch, while the source buffer already contains their full ordered text. One chat bubble grows without duplicated messages. This reduces paint pressure without discarding any content.
Arrival and paint are related but separate clocks. The same buffer survives cancellation or failure. That separation keeps the answer responsive while preserving exact order, which sets up the final problem of stopping safely.
Stop and recover
We have a growing message, but the consumer can pause or disappear. A fast client drains each queue, while a stalled client lets pending chunks accumulate.
Web stream pipe chains can propagate backpressure toward their underlying source. The application server must still respect blocked writes and avoid reading upstream without bounds.
The user closes the answer while the provider still generates. Is waiting for buffers to fill the same signal as cancelling unwanted work?
Stalling fills queues and sends pressure backward only through cooperative layers. Cancellation instead crosses the application boundary and terminates the upstream request deliberately.
Switch Client state through Flowing, Stalled, and Cancelled. Compare the drained queues, accumulated work, and crossed-out upstream path until every outcome is clear.
HTTP error statuses require an explicit response check, and a body read can reject after streaming begins. Preserve partial text, mark terminal state, clean up readers, and retry only when safe. Now let us step back and see the whole picture together.
Recap
We started with the request boundary, where a trusted application server kept the provider key away from public browser code.
Then we learned that typed provider events separate lifecycle state from text deltas, so handlers do not guess from answer content.
SSE framing gave those events blank-line boundaries, while buffering protected them from arbitrary network cuts.
The browser pipeline decoded bytes, retained partial text, parsed only complete frames, and dispatched typed event objects.
Incremental rendering appended every delta to one message buffer, then scheduled bounded paint work for the growing chat bubble.
Finally, pressure, cancellation, and errors produced different terminal paths. Each layer had to propagate the right signal deliberately.
All six mechanisms now form one contract from provider to pixels. The event names and wire rules came from current primary documentation, while our short text fragments were illustrative.
The whole story in 6 lines
A reliable LLM stream preserves boundaries, order, ownership, flow signals, and terminal state from provider to pixels.
- A trusted application server keeps provider credentials away from the browser while relaying the stream.
- Providers emit typed lifecycle events, and only delta events contribute new text to the answer.
- SSE boundaries come from blank lines, not from the arbitrary chunks returned by a network reader.
- Browser code decodes bytes, preserves partial text, parses complete frames, and dispatches typed events.
- A single message buffer preserves order while scheduled paint batches keep rendering work bounded.
- Pressure is local until each layer propagates it, while cancellation requires an explicit upstream signal.