Skip to content
WP EngineDocumentation

Stream a chat completion

Set "stream": true on a chat completion request to receive the response as Server-Sent Events (SSE) instead of a single JSON payload, so you can render tokens to the UI as they’re produced. SSE chunks don’t arrive aligned on event boundaries — a single read may contain several complete data: lines plus a partial line, or just a fragment of one — so the steps below read the stream incrementally and only parse complete lines.

Prerequisites

Section titled “Prerequisites”
  • An API key with Chat set to Write; Full access includes it. Either key type works. An Account key (wpe_) suits a backend service because it doesn’t depend on one person’s access.
  • The key in an AI_API_KEY environment variable on the server that runs this code. Never ship the key to a browser.
  • For the JavaScript example, Node.js 18 or later. For the PHP example, PHP 8.0 or later and Guzzle.

1. Request the stream

Section titled “1. Request the stream”

Set stream: true in the request body. The response comes back with Content-Type: text/event-stream, and the body arrives as a sequence of data: { ... } events instead of a single JSON object.

const BASE_URL = "https://api.ai.wpengine.com";


const res = await fetch(`${BASE_URL}/v1/chat/completions`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.AI_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "google/gemini-3.5-flash",
    stream: true,
    messages: [{ role: "user", content: prompt }],
  }),
});


if (!res.ok || !res.body) {
  throw new Error(`stream request failed: ${res.status}`);
}

2. Read the response incrementally

Section titled “2. Read the response incrementally”

Read the body as it arrives instead of waiting for it to complete.

Each read may return zero or more complete lines plus a trailing partial line — buffer that remainder and prepend it to the next read. The streaming TextDecoder preserves multi-byte UTF-8 sequences across chunk boundaries, so decode with it rather than converting each chunk independently.

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = "";


while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });


  // continue to step 3 with `buffer`
}

3. Parse SSE lines and detect completion

Section titled “3. Parse SSE lines and detect completion”

For each complete line: skip anything that doesn’t start with data: ; treat the sentinel data: [DONE] as the end of the stream; otherwise parse the JSON chat.completion.chunk and hand it to the caller. If the stream fails after it has started, the gateway sends a chunk with an error field instead, followed by [DONE] — throw on it so a failed response isn’t mistaken for a complete one. See streaming errors.

Split the buffer on newlines, holding back the last (possibly incomplete) element for the next read.

const lines = buffer.split("\n");
buffer = lines.pop() ?? "";


for (const line of lines) {
  if (!line.startsWith("data: ")) continue;
  const data = line.slice(6).trim();
  if (data === "[DONE]") return;
  const chunk = JSON.parse(data);
  if (chunk.error) throw new Error(chunk.error.message);
  onDelta(chunk);
}

4. Call it and handle each delta

Section titled “4. Call it and handle each delta”

Steps 1–3 combine into one function that takes a prompt and a callback. If the gateway sends an error chunk, or the body ends without data: [DONE], the function throws rather than returning a partial answer as if it were complete. This example writes each delta to stdout as it arrives.

const BASE_URL = "https://api.ai.wpengine.com";


async function chatStream(prompt, onDelta) {
  const res = await fetch(`${BASE_URL}/v1/chat/completions`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.AI_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: "google/gemini-3.5-flash",
      stream: true,
      messages: [{ role: "user", content: prompt }],
    }),
  });


  if (!res.ok || !res.body) {
    throw new Error(`stream request failed: ${res.status}`);
  }


  const reader = res.body.getReader();
  const decoder = new TextDecoder();
  let buffer = "";


  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    buffer += decoder.decode(value, { stream: true });


    const lines = buffer.split("\n");
    buffer = lines.pop() ?? "";


    for (const line of lines) {
      if (!line.startsWith("data: ")) continue;
      const data = line.slice(6).trim();
      if (data === "[DONE]") return;
      const chunk = JSON.parse(data);
      if (chunk.error) throw new Error(chunk.error.message);
      onDelta(chunk);
    }
  }


  throw new Error("stream ended before [DONE]");
}


await chatStream("Explain how a CDN works in about 100 words.", (chunk) => {
  process.stdout.write(chunk.choices?.[0]?.delta?.content ?? "");
});

Save it as stream.mjs and run node stream.mjs.

5. Observe the output

Section titled “5. Observe the output”

Run the code from step 4 and watch stdout: tokens should print incrementally as the model produces them, not all at once at the end. If nothing appears until the whole response completes, the stream is being buffered somewhere in your stack — check for a proxy or middleware that buffers responses, and confirm you’re reading the body incrementally (res.body.getReader(), or Utils::readLine() with Guzzle’s stream option) rather than awaiting the full body first.

Streaming from WordPress

Section titled “Streaming from WordPress”

WordPress’s HTTP API does not support Server-Sent Events, and the WordPress AI Client does not expose a streaming method. To stream tokens into a WordPress site, run code like the examples above in a service outside the WordPress request, and push tokens to the browser over your own channel.

Last updated: