Streaming responses

SSE framing

Streaming text operations return Content-Type: text/event-stream. Parse complete Server-Sent Events; a TCP chunk is not an event boundary and can contain a partial event or several events.

const response = await fetch(process.env.GETRUN_BASE_URL + '/responses', {
  method: 'POST',
  signal: controller.signal,
  headers: {
    Authorization: `Bearer ${process.env.GETRUN_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ model: 'YOUR_MODEL', input: 'Hello', stream: true }),
});

if (!response.ok || !response.body) throw new Error(`HTTP ${response.status}`);

Use an SSE parser or buffer until a complete blank-line-delimited event is available. Do not call JSON.parse on each raw network chunk.

Cancel a request

Use AbortController to cancel the HTTP request when the user stops generation. If the connection ends unexpectedly, the original request may still have completed. Check usage before retrying to avoid duplicate output and charges.