Streaming Responses

Response streaming delivers AI-generated text token-by-token as the LLM produces it, rather than waiting for the entire response to complete. This creates a more interactive experience where users see text appear in real-time.

Interlocute uses Server-Sent Events (SSE), a standard HTTP protocol for real-time data streaming. When you enable streaming, the platform opens an SSE connection and sends tokens to your client as the LLM generates them.

No. Interlocute uses SSE over HTTP, which works through standard HTTP connections and firewalls. There's no WebSocket setup, no connection management, and no custom protocols. Streaming works with a single API parameter.

Yes. When the node invokes tools or retrieves context from RAG, streaming continues. You receive updates about tool execution, RAG lookups, and the final response as they happen, giving users full visibility into the node's activity.

Yes. Interlocute's embedded chat widgets support streaming by default. Users see AI responses appear in real-time without additional configuration. The same streaming protocol works for iframe embeds, JavaScript widgets, and API integrations.

Streaming reduces perceived latency significantly. Instead of waiting 5-10 seconds for a full response, users see the first tokens in under a second. This makes the AI feel more responsive even though total generation time is the same.

Yes. SSE includes built-in reconnection logic and error handling. If a connection drops, the protocol automatically reconnects and resumes streaming. Interlocute's implementation is production-tested for reliability.

Streaming has no additional cost. Whether you use streaming or wait for the full response, you pay the same per-token price. Streaming is a delivery mechanism, not a separate feature.