Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions .claude/skills/writing-agent-relay-workflows/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -344,6 +344,87 @@ Even if a wave has 10 ready steps, the runner will only start 5 at a time and pi
| 5–10 | 5 |
| 10+ | 6–8 max |

## Scrub Steps: Taming Interactive Lead Output

Interactive agents (PTY mode) produce massive output — TUI chrome, progress indicators, tool call
traces, and verbose reasoning. A single interactive lead step can produce 100KB–10MB of captured
output. When this gets chained via `{{steps.lead-step.output}}` into downstream steps, it:

1. **Overwhelms downstream agents** — they get a 240KB prompt and spiral
2. **Causes timeouts** — the next lead step tries to process megabytes of PTY noise
3. **Breaks verification** — the token exists somewhere in 10MB of output but the agent never exits

### The fix: deterministic scrub steps

After every interactive lead step that produces structured output, add a **deterministic scrub step**
that extracts the clean artifact. Have the lead write its output to a file, then the scrub step
reads that file. Downstream steps chain from the scrub step, not the lead.

```yaml
# Lead writes structured output to a file
- name: propose-boundary
type: agent
agent: lead
dependsOn: [analyze]
task: |
Write your boundary proposal to packages/BOUNDARY.md with this structure:
# Package Boundary
## Package A owns
- ...
## Package B owns
- ...
When done, run: echo "BOUNDARY_PROPOSED"
verification:
type: output_contains
value: BOUNDARY_PROPOSED

# Deterministic step reads the clean file — no PTY noise
- name: scrub-proposal
type: deterministic
dependsOn: [propose-boundary]
command: |
if [ -f packages/BOUNDARY.md ]; then
cat packages/BOUNDARY.md
else
echo "File not found"
fi
captureOutput: true

# Downstream steps chain from the scrub, not the lead
- name: review
agent: reviewer
dependsOn: [scrub-proposal]
task: |
Review this boundary:
{{steps.scrub-proposal.output}}
```

### Rules

- **Every interactive lead step should write its deliverable to a file** (markdown, JSON, etc.)
- **Add a scrub step after each lead step** that `cat`s the file
- **Downstream steps depend on the scrub step**, never directly on the lead step's output
- **Keep lead tasks focused**: one deliverable per step, 10–15 line prompts
- **Non-interactive agents don't need scrub steps** — their stdout is already clean

### When NOT to scrub

- Non-interactive agents (`interactive: false`, presets `worker`/`reviewer`/`analyst`) produce clean
stdout. Their output is safe to chain directly.
- If the lead's output is only consumed by a non-interactive worker (which will read a file anyway),
you can skip the scrub and have the worker read the file directly in its task.

## Verification: Interactive vs Non-Interactive

| Agent type | Verification | Rationale |
|---|---|---|
| Non-interactive (codex workers, `interactive: false`) | None needed | They exit when done — exit code 0 = success |
| Interactive lead (claude PTY) | `output_contains` with `echo "TOKEN"` | Must signal completion explicitly since they stay alive |

Non-interactive agents just die when their task is finished. The runner already treats a clean
exit as success. Adding `output_contains` verification to non-interactive agents triggers the
double-occurrence rule and causes spurious failures. Only use verification on interactive agents.

## Common Mistakes

| Mistake | Fix |
Expand All @@ -361,6 +442,8 @@ Even if a wave has 10 ready steps, the runner will only start 5 at a time and pi
| Workers depending on the lead step (deadlock) | Workers and lead both depend on a shared context step |
| Omitting `agents` field for deterministic-only workflows | Field is now optional — pure shell pipelines work without it |
| Verification token buried at end of long codex task | Use `REQUIRED: The very last line you print must be exactly: TOKEN` |
| Chaining interactive lead output to downstream steps | Add a scrub step that reads the lead's file output (see above) |
| Adding `output_contains` to non-interactive agents | Don't — they exit when done, no verification needed |

## Verification Tokens with Non-Interactive Workers

Expand Down
153 changes: 153 additions & 0 deletions .trajectories/active/traj_1772725458124_28d08389.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
{
"id": "traj_1772725458124_28d08389",
"version": 1,
"task": {
"title": "package-split-design run #49ece71d",
"source": {
"system": "workflow-runner",
"id": "49ece71db0bf6e8626bf8a12"
}
},
"status": "active",
"startedAt": "2026-03-05T15:44:18.124Z",
"agents": [
{
"name": "orchestrator",
"role": "workflow-runner",
"joinedAt": "2026-03-05T15:44:18.124Z"
},
{
"name": "relaycast-analyst",
"role": "relaycast-analyst",
"joinedAt": "2026-03-05T15:44:21.066Z"
},
{
"name": "relay-analyst",
"role": "relay-analyst",
"joinedAt": "2026-03-05T15:44:21.066Z"
},
{
"name": "lead",
"role": "lead",
"joinedAt": "2026-03-05T15:48:26.527Z"
}
],
"chapters": [
{
"id": "ch_2bc81740",
"title": "Planning",
"agentName": "orchestrator",
"startedAt": "2026-03-05T15:44:18.124Z",
"events": [
{
"ts": 1772725458124,
"type": "note",
"content": "Purpose: Multi-agent workflow to split OpenClaw integration into two packages with clear boundaries: @relaycast/openclaw (SDK/bridge layer) and @agent-relay/openclaw (orchestration layer). Uses reflection and consensus to validate the split design."
},
{
"ts": 1772725458124,
"type": "note",
"content": "Approach: 12-step dag workflow — Parsed 12 steps, 3 parallel tracks, 9 dependent steps, DAG validated, no cycles"
}
],
"endedAt": "2026-03-05T15:44:21.025Z"
},
{
"id": "ch_7cb52eae",
"title": "Execution: capture-relay-openclaw, capture-relaycast-openclaw, capture-shared-imports",
"agentName": "orchestrator",
"startedAt": "2026-03-05T15:44:21.025Z",
"events": [],
"endedAt": "2026-03-05T15:44:21.065Z"
},
{
"id": "ch_d666b03a",
"title": "Convergence: capture-relay-openclaw + capture-relaycast-openclaw + capture-shared-imports",
"agentName": "orchestrator",
"startedAt": "2026-03-05T15:44:21.065Z",
"events": [
{
"ts": 1772725461065,
"type": "reflection",
"content": "capture-relay-openclaw + capture-relaycast-openclaw + capture-shared-imports resolved. 3/3 steps completed. All steps completed on first attempt. Unblocking: analyze-relaycast, analyze-relay.",
"significance": "high",
"raw": {
"confidence": 0.75,
"focalPoints": [
"capture-relay-openclaw: completed",
"capture-relaycast-openclaw: completed",
"capture-shared-imports: completed"
]
}
}
],
"endedAt": "2026-03-05T15:44:21.065Z"
},
{
"id": "ch_a6dcb9af",
"title": "Execution: analyze-relaycast, analyze-relay",
"agentName": "orchestrator",
"startedAt": "2026-03-05T15:44:21.065Z",
"events": [
{
"ts": 1772725461066,
"type": "note",
"content": "\"analyze-relaycast\": Analyze the @relaycast/openclaw package and determine what it should own",
"raw": {
"agent": "relaycast-analyst"
}
},
{
"ts": 1772725461066,
"type": "note",
"content": "\"analyze-relay\": Analyze the @agent-relay/openclaw package and determine what should stay",
"raw": {
"agent": "relay-analyst"
}
},
{
"ts": 1772725509011,
"type": "finding",
"content": "\"analyze-relaycast\" completed → - Breaking-change list for migration planning",
"significance": "medium"
},
{
"ts": 1772725706524,
"type": "finding",
"content": "\"analyze-relay\" completed → 1. **Files/functions that should STAY in `@agent-relay/openclaw`**\n\n- [gateway.ts](/Users/khaliqgant/Projects/agent-work",
"significance": "medium"
}
],
"endedAt": "2026-03-05T15:48:26.526Z"
},
{
"id": "ch_c46366e9",
"title": "Convergence: analyze-relaycast + analyze-relay",
"agentName": "orchestrator",
"startedAt": "2026-03-05T15:48:26.526Z",
"events": [
{
"ts": 1772725706527,
"type": "reflection",
"content": "analyze-relaycast + analyze-relay resolved. 2/2 steps completed. All steps completed on first attempt. Unblocking: propose-boundary.",
"significance": "high",
"raw": {
"confidence": 0.75,
"focalPoints": [
"analyze-relaycast: completed",
"analyze-relay: completed"
]
}
},
{
"ts": 1772725706528,
"type": "note",
"content": "\"propose-boundary\": Based on both analyses, assign each module/function/type to a package",
"raw": {
"agent": "lead"
}
}
]
}
]
}
40 changes: 40 additions & 0 deletions .trajectories/active/traj_1772727552183_186931da.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
{
"id": "traj_1772727552183_186931da",
"version": 1,
"task": {
"title": "package-split-design run #57569635",
"source": {
"system": "workflow-runner",
"id": "57569635fe37281929895758"
}
},
"status": "active",
"startedAt": "2026-03-05T16:19:12.183Z",
"agents": [
{
"name": "orchestrator",
"role": "workflow-runner",
"joinedAt": "2026-03-05T16:19:12.183Z"
}
],
"chapters": [
{
"id": "ch_dc0b4159",
"title": "Planning",
"agentName": "orchestrator",
"startedAt": "2026-03-05T16:19:12.183Z",
"events": [
{
"ts": 1772727552183,
"type": "note",
"content": "Purpose: Multi-agent workflow to split OpenClaw integration into two packages with clear boundaries: @relaycast/openclaw (SDK/bridge layer) and @agent-relay/openclaw (orchestration layer). Uses reflection and consensus to validate the split design."
},
{
"ts": 1772727552183,
"type": "note",
"content": "Approach: 12-step dag workflow — Parsed 12 steps, 3 parallel tracks, 9 dependent steps, DAG validated, no cycles"
}
]
}
]
}
49 changes: 49 additions & 0 deletions .trajectories/active/traj_5aodeh2qd7mw.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
{
"id": "traj_5aodeh2qd7mw",
"version": 1,
"task": {
"title": "Analyze @agent-relay/openclaw split into @relaycast/openclaw"
},
"status": "active",
"startedAt": "2026-03-05T15:02:48.290Z",
"agents": [
{
"name": "default",
"role": "lead",
"joinedAt": "2026-03-05T15:03:48.105Z"
}
],
"chapters": [
{
"id": "chap_7jb2vec666xc",
"title": "Work",
"agentName": "default",
"startedAt": "2026-03-05T15:03:48.105Z",
"events": [
{
"ts": 1772723028106,
"type": "decision",
"content": "Move identity/auth/runtime config helpers used by both process+docker setup into @relaycast/openclaw: Move identity/auth/runtime config helpers used by both process+docker setup into @relaycast/openclaw",
"raw": {
"question": "Move identity/auth/runtime config helpers used by both process+docker setup into @relaycast/openclaw",
"chosen": "Move identity/auth/runtime config helpers used by both process+docker setup into @relaycast/openclaw",
"alternatives": [],
"reasoning": "Eliminates duplicate setup primitives and keeps @agent-relay/openclaw focused on WS pairing, spawn lifecycle, MCP/control API"
},
"significance": "high"
}
]
}
],
"commits": [],
"filesChanged": [],
"projectId": "/Users/khaliqgant/Projects/agent-workforce/relay",
"tags": [],
"_trace": {
"startRef": "8033eb28396d49a25ca0215659aed9cac567ea78",
"endRef": "8033eb28396d49a25ca0215659aed9cac567ea78"
}
}78",
"endRef": "8033eb28396d49a25ca0215659aed9cac567ea78"
}
}
Comment on lines +46 to +49
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Malformed JSON in trajectory file causes parse failures

.trajectories/active/traj_5aodeh2qd7mw.json contains duplicated/corrupted content after the valid JSON closing brace. Lines 46-49 are garbage data (}78",\n "endRef": ...) appended after the file's valid end at line 45. Any code that reads this file via JSON.parse() (e.g., packages/trajectory/src/integration.ts:122) will throw a parse error. The readSingleTrajectoryIndex function has a try/catch that would gracefully handle index failures, but individual trajectory reads elsewhere may not be similarly guarded.

Suggested change
}78",
"endRef": "8033eb28396d49a25ca0215659aed9cac567ea78"
}
}
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

42 changes: 42 additions & 0 deletions .trajectories/active/traj_e5she5kpayp2.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"id": "traj_e5she5kpayp2",
"version": 1,
"task": {
"title": "Plan package-boundary migration for @relaycast/openclaw and @agent-relay/openclaw"
},
"status": "active",
"startedAt": "2026-03-05T20:22:10.950Z",
"agents": [
{
"name": "khaliqgant",
"role": "lead",
"joinedAt": "2026-03-05T20:22:10.952Z"
}
],
"chapters": [
{
"id": "chap_59076e8ldxhw",
"title": "Work",
"agentName": "default",
"startedAt": "2026-03-05T20:28:32.975Z",
"events": [
{
"ts": 1772742512977,
"type": "decision",
"content": "Use new workspace package path packages/relaycast-openclaw for @relaycast/openclaw primitives: Use new workspace package path packages/relaycast-openclaw for @relaycast/openclaw primitives",
"raw": {
"question": "Use new workspace package path packages/relaycast-openclaw for @relaycast/openclaw primitives",
"chosen": "Use new workspace package path packages/relaycast-openclaw for @relaycast/openclaw primitives",
"alternatives": [],
"reasoning": "Keeps existing packages/openclaw as @agent-relay/openclaw while enabling one-way dependency and explicit split ownership"
},
"significance": "high"
}
]
}
],
"commits": [],
"filesChanged": [],
"projectId": "/Users/khaliqgant/Projects/agent-workforce/relay",
"tags": []
}
Loading
Loading