xterm.js web terminal component for agent sessions

[geoffjay]

Context

Part of #671 (PTY execution backend epic). Blocked by: #675 (PTY stream relay endpoint).

Add a web terminal component using xterm.js that connects to the orchestrator's /terminal/{agent_id} endpoint, providing a real terminal experience for agent sessions in the UI.

Problem

The current AgentLogView.tsx strips ANSI sequences and displays plain timestamped log lines. This is great for structured viewing but loses the rich terminal experience. Users want to:

• See Claude's output as it would appear in a real terminal (colors, formatting, progress indicators)
• Optionally interact with the session (send keystrokes, Ctrl-C)
• Debug agent behavior with full terminal fidelity

Design

Component: AgentTerminal.tsx

A new component that renders an xterm.js terminal connected to the PTY stream relay.

interface AgentTerminalProps {
  agentId: string;
  readOnly?: boolean;  // default: true (observation mode)
}

Tab Layout

The agent detail page gets two tabs:

• Logs (default) — existing AgentLogView.tsx (structured, searchable, timestamped)
• Terminal — new AgentTerminal.tsx (raw PTY, full terminal rendering)

The Terminal tab should show a badge/indicator when the backend doesn't support PTY streaming (tmux/docker backends).

Features

1. Terminal rendering: Full xterm.js with ANSI color support, cursor positioning, scrollback
2. Auto-connect: Connects to ws://{orchestrator}/terminal/{agentId} when tab is active
3. Read-only mode (default): Displays output but doesn't send input
4. Interactive mode (opt-in toggle): Forwards keyboard input to PTY
5. Fit addon: Auto-resize terminal to container, send resize messages
6. Reconnection: Match existing WebSocket reconnection behavior (exponential backoff)
7. Copy/paste: Standard terminal selection and clipboard support
8. Theme: Match the existing UI dark/light theme

Dependencies

{
  "@xterm/xterm": "^5.x",
  "@xterm/addon-fit": "^0.x",
  "@xterm/addon-web-links": "^0.x",
  "@xterm/addon-search": "^0.x"
}

Implementation Steps

1. Install xterm.js and addons: bun add @xterm/xterm @xterm/addon-fit @xterm/addon-web-links @xterm/addon-search
2. Create ui/src/components/agents/AgentTerminal.tsx
3. Implement WebSocket connection to /terminal/{agent_id} with binary frame handling
4. Add fit addon for auto-resize with resize message sending
5. Add read-only / interactive mode toggle
6. Create tab layout in agent detail page (Logs | Terminal)
7. Handle graceful degradation when PTY backend isn't available
8. Style to match existing UI theme
9. Add search capability (addon-search)

Relevant Files

• ui/src/components/agents/AgentLogView.tsx — existing log display (reference)
• ui/src/services/websocket.ts — existing WebSocket manager (reference for connection patterns)
• ui/src/hooks/useAgentDetail.ts — agent detail page hook
• ui/src/pages/AgentsPage.tsx — agents page layout

Acceptance Criteria

• xterm.js terminal renders agent PTY output with full ANSI support
• Terminal auto-resizes to container and sends resize events
• Tab switcher between Logs and Terminal views
• Read-only mode by default, interactive mode toggle
• WebSocket reconnection with exponential backoff
• Graceful fallback when PTY streaming unavailable (message explaining tmux backend limitation)
• Theme matches existing UI (dark/light)
• Copy/paste works with standard terminal selection
• No performance degradation for long-running sessions (xterm.js scrollback limits)

Caution

All work for this issue must branch from feature/autonomous-pipeline and PR back into it — never directly to main.
Use bun (not npm) for all package management.

[geoffjay]

🔍 Issue Enrichment

Added by the enricher agent to help clarify scope and guide implementation.

Context from memory: The planner noted that #676 should route through design-agent first, then agent for implementation. This issue already carries the design-agent label — this enrichment is positioned for the implementer once the design phase is complete.

Acceptance Criteria

• ui/src/components/agents/AgentTerminal.tsx is created and exported from ui/src/components/agents/index.ts
• AgentDetail.tsx tab switcher replaces the bare <div className="h-[480px]"> wrapping AgentLogView with a Logs / Terminal tab bar; the AgentLogView remains the default tab and its existing props/behavior are unchanged
• AgentTerminal connects to ws://{orchestratorBase}/terminal/{agentId} using WebSocketManager from ui/src/services/websocket.ts — reusing existing exponential backoff (1 s → 30 s) and heartbeat handling rather than implementing a new WebSocket layer
• The AgentTerminal component passes incoming message data directly to terminal.write() rather than buffering in React state — xterm.js maintains its own scrollback buffer; React state must not hold terminal output
• WebSocket messages from the PTY relay are handled as binary (ArrayBuffer / Blob) or UTF-8 text; the handler must convert Blob via blob.arrayBuffer() before calling terminal.write() since the existing WebSocketManager.send() only enqueues strings but onMessage delivers raw MessageEvent data
• The @xterm/addon-fit ResizeObserver triggers fitAddon.fit() on container resize — not only on mount — and sends a JSON resize message { type: "resize", cols, rows } to the relay endpoint after each fit
• Terminal is only connected when the Terminal tab is active (paused pattern from useWebSocket); disconnects when the Logs tab is active to avoid consuming relay resources in the background
• xterm.js Terminal instance is created once in a useEffect with a DOM ref, opened with terminal.open(containerRef.current), and disposed with terminal.dispose() on unmount — not re-created on re-render
• Theme is sourced from useTheme (ui/src/hooks/useTheme.tsx) and applied to the xterm.js theme option at construction time; theme changes cause the terminal to re-initialize with updated colors
• Read-only mode (default): keyboard input is suppressed via terminal.onKey returning without calling send()
• Interactive mode toggle: visible in the toolbar; when enabled, terminal.onKey forwards keystrokes as binary/UTF-8 frames to the relay; toggle is disabled (grayed) when the agent is not running
• Graceful degradation: when the WebSocket connection is refused or returns a non-101 response (PTY backend unavailable), display a styled callout explaining that the Terminal tab requires the PTY backend (AGENTD_BACKEND=pty) rather than a blank or spinning terminal
• Scrollback capped at a configurable limit (default 5 000 lines, matching MAX_LINES in useAgentStream) to prevent memory growth for long-running sessions — passed as scrollback to xterm.js Terminal constructor
• All four xterm.js packages (@xterm/xterm, @xterm/addon-fit, @xterm/addon-web-links, @xterm/addon-search) installed via bun add (not npm)
• Tests use vi.mock('@xterm/xterm') (vitest) since xterm.js requires a real DOM not available in jsdom; the test file follows the pattern in AgentLogView.test.tsx

Relevant Files

• ui/src/pages/agents/AgentDetail.tsx:769–772 — the <div className="h-[480px]"> wrapping AgentLogView is exactly where the Logs/Terminal tab switcher is introduced; the tab state ('logs' | 'terminal') lives here as local useState
• ui/src/components/agents/AgentLogView.tsx — reference for toolbar layout (StreamStatusBadge, clear button, toggle buttons), scroll-lock pattern, and aria-live region; AgentTerminal toolbar should follow the same structure
• ui/src/services/websocket.ts — WebSocketManager class; use directly (not via useWebSocket) to avoid buffering MessageEvent[] in React state — call terminal.write() in the onMessage handler instead
• ui/src/hooks/useWebSocket.ts — shows the paused option that suppresses auto-connect; replicate this pattern in AgentTerminal to avoid connecting when the Logs tab is active
• ui/src/hooks/useAgentStream.ts — reference for agentStreamUrl() helper pattern; write a parallel agentTerminalUrl(agentId) helper that substitutes /stream/ with /terminal/
• ui/src/hooks/useTheme.tsx — provides current theme; map dark/light to an xterm.js IThemeOptions object for the Terminal constructor
• ui/src/components/agents/index.ts — add export { AgentTerminal } from './AgentTerminal' here

Related Issues

• Add PTY stream relay endpoint to orchestrator #675 — PTY stream relay endpoint (orchestrator) — blocks this issue; AgentTerminal connects to /terminal/{agentId} which is implemented in Add PTY stream relay endpoint to orchestrator #675; the WebSocket protocol (binary vs text frames, resize message format) must be agreed between Add PTY stream relay endpoint to orchestrator #675 and this issue
• Epic: PTY execution backend and web terminal support #671 — Parent epic — xterm.js is the last frontend piece in the dependency chain
• Backend selection configuration and documentation #678 — Configuration & docs — should document the AGENTD_BACKEND=pty requirement for the Terminal tab
• design: pipeline status card and human gates UI for v0.9.0 milestone #18 #648 — Prior design PR referenced in memory for PipelineStatusCard — not directly related but confirms the UI codebase uses stub patterns for backend-dependent features; the graceful-degradation callout in this issue follows the same principle

Suggested Test Cases

1. Renders terminal container: render(<AgentTerminal agentId="abc" />) — container div with aria-label="Agent terminal" is present in the DOM; xterm.js Terminal constructor is called once
2. Does not connect when paused: Render with active={false} prop (Logs tab showing) — WebSocketManager.connect() is NOT called; verify via mock
3. Connects when active: Toggle to Terminal tab — WebSocketManager.connect() is called; verify the URL includes /terminal/abc
4. Writes binary data to terminal: Simulate a MessageEvent with ArrayBuffer data — verify terminal.write() is called with the correct Uint8Array
5. Read-only by default: Keyboard events inside the terminal do NOT call ws.send() — verify via mock
6. Interactive mode sends keystrokes: Toggle interactive mode ON, simulate a key event — ws.send() is called with the keystroke data
7. Graceful degradation: WebSocketManager fires 'Disconnected' state on initial connect — terminal area shows the PTY-unavailable callout, not a spinner
8. Edge case — dispose on unmount: terminal.dispose() and ws.disconnect() are both called when the component unmounts; verify no memory leak via mock call counts

Implementation Hints

• Do not use useWebSocket hook — it buffers MessageEvent[] in React state (capped at 200), which is wrong for terminal output. Instantiate WebSocketManager directly in a useEffect, call terminal.write() inside onMessage, and clean up with manager.disconnect() on the effect teardown. See useAgentStream for this exact pattern.
• terminal.open() must be called after the ref's DOM node is available — use useEffect(() => { if (!containerRef.current) return; ... }, []) with terminal.dispose() in the cleanup.
• Blob handling: Browser WebSocket may deliver binary PTY data as Blob (not ArrayBuffer). Guard: if (event.data instanceof Blob) { event.data.arrayBuffer().then(buf => terminal.write(new Uint8Array(buf))) } else if (event.data instanceof ArrayBuffer) { terminal.write(new Uint8Array(event.data)) } else { terminal.write(event.data) }
• fitAddon.fit() after open(): Call fitAddon.fit() in a requestAnimationFrame callback after terminal.open() to ensure the container has been painted and has a non-zero size before the fit calculation runs.
• Tab state in AgentDetail: Add const [activeTab, setActiveTab] = useState<'logs' | 'terminal'>('logs') near the other useState calls (around line 480–500 in AgentDetail.tsx). Pass active={activeTab === 'terminal'} to AgentTerminal.
• xterm.js mock for tests: vi.mock('@xterm/xterm', () => ({ Terminal: vi.fn(() => ({ open: vi.fn(), write: vi.fn(), dispose: vi.fn(), onKey: vi.fn(), loadAddon: vi.fn() })) })) — place in the test file, not a shared setup, since other tests don't need it.