Skip to content

SSE streaming for real-time agent execution updates#610

Open
fractlrao wants to merge 3 commits intomainfrom
sse-events-for-agent-updates
Open

SSE streaming for real-time agent execution updates#610
fractlrao wants to merge 3 commits intomainfrom
sse-events-for-agent-updates

Conversation

@fractlrao
Copy link
Copy Markdown
Contributor

Summary

  • Adds Server-Sent Events (SSE) streaming so API clients receive real-time progress updates during agent execution
  • Clients opt in by sending Accept: text/event-stream on POST event endpoints — non-streaming requests are completely unchanged
  • Event types are plain strings (not a fixed enum), so any part of the runtime can emit custom events on the fly
  • Includes unit tests for the SSE emitter and environment integration, plus an end-to-end integration test

Use cases

Chat UIs and dashboards — Instead of showing a blank spinner for 10-30 seconds while an agent runs, a frontend can display live progress: which flow step is executing, when the LLM responds, and intermediate results as they arrive.

Multi-step agent flows — When an agent orchestrates a flow with multiple steps (data retrieval, decision-making, API calls), each step transition streams as a flow_step_start/flow_step_complete event. A client can render a live step-by-step timeline.

Debugging and observability — Developers building Agentlang apps can curl -N with the SSE header and watch the full execution trace in real time, seeing exactly where time is spent and where errors occur.

Error handling mid-stream — If an LLM call fails partway through a multi-step flow, the client receives the error event immediately rather than waiting for the full timeout, enabling faster user feedback and retry logic.

Long-running agent tasks — For agents that perform complex planning with retries, the client stays informed throughout instead of facing an opaque wait.

How it works

  • New SseEmitter class (src/api/sse.ts) wraps an Express Response for SSE protocol: headers, keep-alive, event formatting, and cleanup
  • Environment gains setSseEmitter() / emitSseEvent() — the emitter propagates from parent to child environments automatically
  • emitSseEvent is null-safe: no-ops when no emitter is set, so zero impact on non-streaming paths
  • Events emitted at key points: agent start/complete, flow step transitions, LLM responses (with token usage), and errors
  • handleEventPost in http.ts forks on the Accept header — SSE path creates an emitter environment, non-SSE path is untouched

Test plan

  • SseEmitter unit tests: headers, event format, arbitrary event types, close idempotency, client disconnect
  • isStreamingRequest tests: various Accept header values
  • Environment.emitSseEvent tests: no-op without emitter, emit with emitter, no-op when closed, child inheritance
  • Integration test: evaluate() with SSE emitter through full runtime bootstrap
  • All 120 existing tests pass with no regressions
  • Manual test: curl -N -H "Accept: text/event-stream" against a running app with an agent event

Clients can opt into Server-Sent Events by sending
Accept: text/event-stream on POST event endpoints. Progress
events (flow steps, agent invocations, LLM responses, errors)
are streamed in real time, with the final result closing the
stream. Non-streaming requests are completely unchanged.
Unit tests for SseEmitter (event format, close behavior, client
disconnect, arbitrary event types) and Environment.emitSseEvent
(null-safety, parent-to-child propagation). Integration test
verifies SSE emitter flows through evaluate with a full runtime
bootstrap.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant