Squad product: Linked chat summaries -- investigation and StorageProvider dependency #101
Description
Scope: This is a Squad product investigation issue. It connects Brady's "Linked chat summaries" feature exploration to the StorageProvider abstraction in PR bradygaster#640.
Background: Brady's "Linked chat summaries" email thread
Brady initiated an email thread exploring linked chat summaries as a new Squad feature. Key points from the discussion (Brady, Paul Hsu, and team):
- Treat linked chat summaries as an experimental, gated feature
- Develop and test in a private fork/branch first
- Use environment variables to gate the feature and tune thresholds
- Run established benchmarks (LOCOMO, LongMemEval, Tau Bench) to validate
- Collect opt-in telemetry from real users (full conversation histories, token usage, memory retrieval)
- Use simulation to compare outcomes when full dialog history is available
- Coordinate release messaging carefully
- Short testing window (a few days) before convergence
Connection to bradygaster#640 (StorageProvider)
PR bradygaster#640 introduces the StorageProvider interface with FSStorageProvider, InMemoryStorageProvider, and SQLiteStorageProvider. This is directly relevant to linked chat summaries because:
-
Chat history persistence -- linked chat summaries need to store and retrieve conversation histories. The StorageProvider abstraction gives this feature a clean persistence layer without coupling to the filesystem.
-
InMemoryStorageProvider for testing -- Paul's benchmarking plan (LOCOMO, LongMemEval, Tau Bench) needs fast, isolated test runs. InMemory provider enables this without disk I/O overhead.
-
SQLiteStorageProvider for telemetry -- opt-in telemetry collection (token usage, memory retrieval metrics) naturally fits a structured storage backend like SQLite rather than flat files.
-
Environment variable gating -- the StorageProvider pattern already supports config-driven selection. The linked chat summaries feature could use a similar pattern:
SQUAD_CHAT_SUMMARIES=enabledselects the right storage backend. -
Private fork development -- Brady wants to develop this in a private fork. The StorageProvider PR (feat(sdk): StorageProvider abstraction — complete migration + example providers bradygaster/squad#640) should land on upstream dev first so the private fork can build on it.
Work to be done
Pre-requisites (must land first)
- Merge feat(sdk): StorageProvider abstraction — complete migration + example providers bradygaster/squad#640 (StorageProvider) to upstream dev -- provides the persistence abstraction
- Strip bleed files from feat(sdk): StorageProvider abstraction — complete migration + example providers bradygaster/squad#640 (5 files identified by Flight's review)
Investigation tasks
- Determine where chat summaries would be stored (which StorageProvider implementation)
- Design the linked summary data model (what gets stored, how summaries link across sessions)
- Define the environment variable gating pattern (on/off switch, threshold tuning)
- Identify which benchmarks (LOCOMO, LongMemEval, Tau Bench) are feasible to run locally
- Design the opt-in telemetry schema (token usage, memory retrieval, conversation length)
Implementation tasks (after investigation)
- Implement chat summary storage using StorageProvider interface
- Add environment variable gates
- Build benchmark harness for comparison testing
- Add telemetry collection (opt-in, using StorageProvider)
- Create private fork branch for development
Process
- Align with Brady and Tamir on scope before cutting private fork
- Set up Teams thread for async coordination (Brady requested this)
- Define short testing window for experimentation
Related
- feat(sdk): StorageProvider abstraction — complete migration + example providers bradygaster/squad#640 -- StorageProvider abstraction (PR on bradygaster/squad, draft)
- Brady's email thread: "Linked chat summaries" (Brady, Paul Hsu)
- Squad product: Default workflows burn too many Actions minutes for multi-repo customers #98 -- Config-driven CI (environment variable gating pattern is shared)