Skip to content

Add protocol layer for requirements, coverage, signals, and pool gap computation#20

Open
DarrenZal wants to merge 5 commits intoregen-prodfrom
feat/requirements-coverage
Open

Add protocol layer for requirements, coverage, signals, and pool gap computation#20
DarrenZal wants to merge 5 commits intoregen-prodfrom
feat/requirements-coverage

Conversation

@DarrenZal
Copy link
Copy Markdown
Contributor

Summary

Additive protocol layer implementing Steps 1-5 of the Claims × Spore coordination protocol. Builds on existing claims/commitments/intents without rewrites.

New protocol tables:

  • requirements (079) — normative expectations with cadence (frequency, freshness window, severity). Generic across pool/org/federation scopes.
  • coverage_links (081) — explicit relational primitive: commitment covers requirement, claim covers condition, evidence covers commitment. Replaces ad-hoc absence inference with deterministic gap computation.
  • signals (082) — raw observations and computed gaps. Junction table links signals → existing intent_registry.

Schema extensions:

  • scope column on commitments and commitment_pools (080) — first-class, not metadata. Determines governance path.

New /protocol/* router:

  • POST /protocol/requirements/create — declare a requirement (upserts mutable fields)
  • GET /protocol/requirements/ — list with scope/type/severity filters
  • POST /protocol/coverage/link — create coverage link (full upsert)
  • GET /protocol/coverage/ — list with validity filtering
  • POST /protocol/signals/create — record a signal
  • GET /protocol/signals/ — list with type/scope/freshness filters
  • GET /protocol/pools/{rid}/gaps — negative-space intelligence: computes unmet/stale requirements via coverage chain, emits gap_computed signals with next-move suggestions (surface_only, request_offer, propose_commitment, escalate_to_council)

Small API extensions:

  • claims_router.py: since datetime filter on list endpoint (422 on malformed input)
  • commitment_router.py: offer_type filter on list endpoint, scope in response models

Demo path (end-to-end test)

1. Create pool
2. Create requirement: "Monthly biodiversity survey" (frequency=monthly, severity=high)
3. GET /protocol/pools/{rid}/gaps → 1 unmet gap, next_move=propose_commitment
4. Verify gap_computed signal emitted
5. Add coverage (commitment covers requirement)
6. GET /protocol/pools/{rid}/gaps → 0 gaps, 1 covered

Test plan

  • 14/14 tests passing (tests/test_protocol_layer.py)
  • Requirement RID uniqueness across pools
  • Requirement upsert updates mutable fields
  • Gap signal recomputation upserts metadata
  • Future-dated coverage does not suppress gap
  • Wrong coverage type does not suppress gap
  • Valid coverage suppresses gap
  • Expired coverage produces stale gap (not unmet)
  • Malformed since returns 422
  • Commitment/pool responses include scope
  • End-to-end gap path with signal emission and resolution
  • Pool not found returns 404
  • Commitment offer_type filter

🤖 Generated with Claude Code

DarrenZal and others added 4 commits March 29, 2026 22:14
Decouple classifier, chat answer, and query expansion from hardcoded
OpenAI client. Each surface now has its own ChatProvider instance
configured via independent env vars (CLASSIFIER_PROVIDER/MODEL,
CHAT_PROVIDER/MODEL, EXPANSION_PROVIDER/MODEL).

- New api/chat_provider.py: ABC + OpenAI/Anthropic implementations +
  3 factory functions. Follows embedding_provider.py pattern.
- Classifier stays OpenAI-only (structured output sensitive).
- Chat answer supports OpenAI and Anthropic.
- Expansion stays OpenAI-only.
- Removed openai_client global from personal_ingest_api.py entirely.
- Fixed _expand_queries sync-in-async bug (now awaits provider).
- Lazy-init with 503 on config error in request path.
- 78 passed, 7 skipped, 0 failed across 5 test files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Double-gated: request.debug_prompt=true AND CHAT_DEBUG_PROMPT env var
must both be set. Returns assembled system_prompt + user_prompt in
response for offline provider comparison. 3 gating tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bakeoff result: OpenAI wins 6/8, stays as default. Anthropic wins
only on dense commitment/mechanism questions (commitment_claim_1,
commitment_claim_3). No on-node Anthropic trial planned.

- scripts/bakeoff_chat_answers.py: local harness that runs both
  providers on frozen prompt packets and generates comparison.md
- debug_prompt flag on /chat (committed in prior P2-Phase1 commit)
  enabled packet capture; CHAT_DEBUG_PROMPT already removed from Octo

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Additive protocol layer on top of existing claims/commitments/intents
(Claims × Spore coordination protocol, Steps 1-5):

- requirements table: normative expectations with cadence (079)
- scope column on commitments/pools (080)
- coverage_links table: relational primitive for gap computation (081)
- signals table: raw observations and computed gaps (082)
- GET /protocol/pools/{rid}/gaps: negative-space intelligence endpoint
- POST /protocol/requirements/create, GET /protocol/requirements/
- POST /protocol/coverage/link, GET /protocol/coverage/
- POST /protocol/signals/create, GET /protocol/signals/
- claims_router: since date filter on list endpoint
- commitment_router: offer_type filter, scope in response models

Gap computation checks coverage_links for each active requirement.
No valid coverage → unmet gap. Expired coverage → stale gap.
Emits gap_computed signals as audit trail with next-move suggestions.

14/14 tests passing (test_protocol_layer.py).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@DarrenZal
Copy link
Copy Markdown
Contributor Author

Quick review guide

The one endpoint to try: GET /protocol/pools/{pool_rid}/gaps

End-to-end flow (6 steps):

  1. Create a pool
  2. Create a requirement scoped to that pool (e.g., "quarterly monitoring", severity=high)
  3. GET /protocol/pools/{pool_rid}/gaps → returns 1 unmet gap with next_move: propose_commitment
  4. A gap_computed signal is emitted as audit trail
  5. Add a coverage link (commitment covers requirement)
  6. GET /protocol/pools/{pool_rid}/gaps → 0 gaps, 1 covered. Gap resolved.

Headline: The system computes what's absent — a requirement exists but no commitment covers it — and surfaces a structured gap signal with a suggested next move.

14/14 tests passing in tests/test_protocol_layer.py, including the full e2e path above.

A standalone demo script (curl sequence) is in a follow-up comment.

@DarrenZal
Copy link
Copy Markdown
Contributor Author

Demo script

scripts/demo_protocol_gaps.sh reproduces the full e2e path:

bash scripts/demo_protocol_gaps.sh http://localhost:8351

Steps: create pool → create requirement (quarterly monitoring, severity=high) → compute gaps (1 unmet, next_move=propose_commitment) → verify signal emitted → add commitment + coverage → recompute gaps (0 unmet, 1 covered).

The script is not committed to the PR branch — it's a standalone artifact for reviewers. The same flow is validated by test_e2e_gap_path in tests/test_protocol_layer.py.

1. Requirement RID now includes requirement_type and subject_uri —
   prevents same-statement different-subject collapse in same pool.
   Signal RID now includes subject_uri for the same reason.

2. Stale-history fallback query now requires valid_from <= now —
   future-dated coverage with a finite end date is classified as
   unmet (upcoming), not stale.

3. Severity ordering uses CASE expression instead of lexical sort —
   critical > high > medium > low instead of alphabetical.

Two new tests: same-pool different-subject RID uniqueness,
future-dated coverage with end date is unmet not stale.
16/16 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant