SSE streaming for real-time agent execution updates#610
Open
SSE streaming for real-time agent execution updates#610
Conversation
Clients can opt into Server-Sent Events by sending Accept: text/event-stream on POST event endpoints. Progress events (flow steps, agent invocations, LLM responses, errors) are streamed in real time, with the final result closing the stream. Non-streaming requests are completely unchanged.
Unit tests for SseEmitter (event format, close behavior, client disconnect, arbitrary event types) and Environment.emitSseEvent (null-safety, parent-to-child propagation). Integration test verifies SSE emitter flows through evaluate with a full runtime bootstrap.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Accept: text/event-streamon POST event endpoints — non-streaming requests are completely unchangedUse cases
Chat UIs and dashboards — Instead of showing a blank spinner for 10-30 seconds while an agent runs, a frontend can display live progress: which flow step is executing, when the LLM responds, and intermediate results as they arrive.
Multi-step agent flows — When an agent orchestrates a flow with multiple steps (data retrieval, decision-making, API calls), each step transition streams as a
flow_step_start/flow_step_completeevent. A client can render a live step-by-step timeline.Debugging and observability — Developers building Agentlang apps can
curl -Nwith the SSE header and watch the full execution trace in real time, seeing exactly where time is spent and where errors occur.Error handling mid-stream — If an LLM call fails partway through a multi-step flow, the client receives the error event immediately rather than waiting for the full timeout, enabling faster user feedback and retry logic.
Long-running agent tasks — For agents that perform complex planning with retries, the client stays informed throughout instead of facing an opaque wait.
How it works
SseEmitterclass (src/api/sse.ts) wraps an ExpressResponsefor SSE protocol: headers, keep-alive, event formatting, and cleanupEnvironmentgainssetSseEmitter()/emitSseEvent()— the emitter propagates from parent to child environments automaticallyemitSseEventis null-safe: no-ops when no emitter is set, so zero impact on non-streaming pathshandleEventPostinhttp.tsforks on theAcceptheader — SSE path creates an emitter environment, non-SSE path is untouchedTest plan
SseEmitterunit tests: headers, event format, arbitrary event types, close idempotency, client disconnectisStreamingRequesttests: various Accept header valuesEnvironment.emitSseEventtests: no-op without emitter, emit with emitter, no-op when closed, child inheritanceevaluate()with SSE emitter through full runtime bootstrapcurl -N -H "Accept: text/event-stream"against a running app with an agent event