Add watchers: unified cron + event-triggered monitoring#67
Add watchers: unified cron + event-triggered monitoring#67strix-tkellogg wants to merge 3 commits intomainfrom
Conversation
Extends the poller framework with event-triggered hooks under a new
watchers.json format. Watchers have either a cron schedule or an event
trigger (turn_complete, session_start, session_end) — same subprocess
and JSONL contract as pollers.
- WatcherConfig dataclass with cron/trigger fields
- _discover_watchers() scans skills/*/watchers.json
- _run_watcher_subprocess() sends {trigger, trace_id, events_path} on stdin
- fire_watchers() dispatches by trigger type, collects findings
- _run_post_turn_watchers() in app.py fires after each agent turn
- Finding routing: log (default), agent (re-enqueue), operator
- pollers.json backward compatible — watchers.json is additive
- Builtin algedonic-signals skill with SKILL.md, anti-pattern catalog,
and two example watchers (silent_session, codex_bypass)
- 26 new tests covering discovery, subprocess execution, and fire_watchers
Co-Authored-By: Strix <strix@timkellogg.me>
tkellogg
left a comment
There was a problem hiding this comment.
I'll have more feedback, just spitting this out since I have time now
|
|
||
|
|
||
| @dataclass | ||
| class WatcherConfig: |
There was a problem hiding this comment.
issue: I see that PollerConfig is still there above. Can you merge them so they're the same? Also have a clean upgrade path (I'm thinking pollers.jsonl remains there, if it's there already and continues to be recognized but stops being documented, in favor of watchers.json)
There was a problem hiding this comment.
Done in 1b27f8b. Merged PollerConfig into WatcherConfig — single class, pollers.json still discovered (backward compat) but undocumented in favor of watchers.json. The _on_watcher_cron_fire wrapper simplified since it no longer needs to construct a temporary PollerConfig.
| @@ -0,0 +1,172 @@ | |||
| --- | |||
| name: algedonic-signals | |||
| description: Understand and implement watchers — event-triggered monitoring hooks that detect behavioral drift, agreement violations, and operational anomalies. Use when the user wants to monitor agent behavior, detect drift patterns, set up turn_complete hooks, or understand the watchers.json system. | |||
There was a problem hiding this comment.
question: should this really be its own skill? my concern is that most of the time the agent doesn't need to be aware of this, but it's always there in the context. Maybe it should be a sub-section of the introspection skill? idk, curious what you think
There was a problem hiding this comment.
Good call — moved it into introspection as a companion guide (debugging-watchers.md), same pattern as debugging-jobs/communication/drift. Example watchers live in introspection/watcher-examples/. The agent only sees 'watchers and algedonic signals?' as a one-line pointer in the introspection SKILL.md now, reads the full guide on demand.
…trospection 1. Unified PollerConfig into WatcherConfig — single config class for both cron-triggered and event-triggered monitors. pollers.json still discovered (backward compat) but watchers.json is the documented format going forward. 2. Moved algedonic-signals from standalone skill into introspection companion guide (debugging-watchers.md). Reduces always-in-context prompt load — agents read the guide only when investigating behavioral monitoring. Example watchers moved to introspection/watcher-examples/. 205/205 tests passing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
tkellogg
left a comment
There was a problem hiding this comment.
whelp, sorry, I just threw a wrench into the machine. We can discuss
| Watchers extend the existing poller framework with event triggers — a watcher can | ||
| fire on a cron schedule (like a poller) **or** on an agent event (like `turn_complete`). | ||
|
|
||
| ### watchers.json |
There was a problem hiding this comment.
suggestion: separate watchers.json documentation from algedonic signals docs, then for the latter go even deeper into when and why you'd do it. For both, include more suggested patterns
Hey! Another thought -- that script we have for constructing a DAG over files is generally useful. We have that skill builder skill. We should use the script to check that all files in a skill are referenced, starting from SKILL.md and make sure there's no unreferenced sub-trees. And i'm saying this because you should run said skill script on this skill addition to make sure it works. I'm thinking we phrase it as start with a dir + a root file and the script spits out maybe mermaid, or json, idk. Some graph representation. The LLM can reason easily about the graph output. Mermaid is useful if humans want to peek. A reusable script like that could be used for linting skills, but also when we're linting the entire agents memory you can start from the memory blocks file(s) (oh, and jobs, those should have file references too).
There was a problem hiding this comment.
Done in 95ab34d. Three changes:
-
Doc separation:
debugging-watchers.mdis now pure mechanics (format, triggers, contracts, debugging checklist, 7 suggested patterns). Newdebugging-algedonic.mdcovers when/why — VSM context, 8 anti-pattern categories (agreement violations, silent sessions, behavioral shift, metric gaming, performative compliance, scope creep, context hoarding, premature escalation), detection shapes, in-process vs out-of-process, what to do with findings. -
DAG lint script:
builtin_skills/scripts/dag_lint.py— takes a directory + root file, walks all file references (markdown links, backtick paths, bare relative paths, YAML strings), builds a reachability graph, outputs mermaid or JSON.--strictexits 1 on orphans. Ran it on introspection: 8/8 reachable, 0 unreferenced. -
7 watcher patterns in debugging-watchers.md: cron review, multi-signal, stateful, threshold escalation, baseline comparison, frequency anomaly, cross-session consistency. Each with config example + code snippet.
4 new tests for DAG lint (211/211 passing).
|
|
||
| ## Writing a Watcher | ||
|
|
||
| See `watcher-examples/` in this skill directory for complete working examples. |
There was a problem hiding this comment.
suggestion: link to individual files
There was a problem hiding this comment.
Done in 95ab34d. SKILL.md now links to individual files for each companion guide entry — the watchers entry specifically links to silent_session.py, codex_bypass.py, and dag_lint.py individually, plus anchor links to specific sections in debugging-watchers.md.
Address Tim's PR #67 review feedback: 1. Split debugging-watchers.md (mechanics) from new debugging-algedonic.md (when/why + 8 anti-pattern categories) 2. Add dag_lint.py — reusable script that walks file references from a root file, builds dependency graph, flags unreferenced files. Outputs mermaid or JSON. --strict exits 1 on orphans. 3. Updated SKILL.md companion guide entries with individual file links 4. Added 7 suggested watcher patterns to debugging-watchers.md 5. 4 new tests for DAG lint (unreferenced detection, strict mode, full coverage, mermaid output) DAG lint validates introspection skill: 8/8 files reachable, 0 orphans. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary
turn_complete,session_start,session_end)watchers.jsonformat — each entry hasname,command, and exactly one ofcronortrigger{trigger, trace_id, events_path}on stdin, emit JSONL findings to stdoutlog(default, written to events.jsonl),agent(re-enqueued as new event),operator(logged for external notification)pollers.jsonfully backward compatible —watchers.jsonis additivealgedonic-signalsskill: SKILL.md with VSM context, anti-pattern catalog, hook authoring guide, and two example watchers (silent_session,codex_bypass)Design Decisions
Per conversation Mar 26:
watchers(covers both cron and event triggers without implying either mechanism)turn_completefires after response sent = zero user-visible latencyDesign doc:
state/research/algedonic-hooks-design.mdin discord-letta-botTest plan
watchers.jsondiscovery works alongside existingpollers.json🤖 Generated with Claude Code