Skip to content

feat(sdk): StateBackend POC - orphan branch persistence [DO NOT MERGE]#680

Open
tamirdresher wants to merge 2 commits intodevfrom
squad/678-git-notes-poc
Open

feat(sdk): StateBackend POC - orphan branch persistence [DO NOT MERGE]#680
tamirdresher wants to merge 2 commits intodevfrom
squad/678-git-notes-poc

Conversation

@tamirdresher
Copy link
Copy Markdown
Collaborator

What

POC: StateBackend storing Squad state in git orphan branch - immune to branch switches and gitignore.

Why

Solves root cause of #643 (state destroyed), #498 (remove .squad/ from VCS), #670 (layout manifest). See RFC #678.

How

New module packages/squad-sdk/src/state/:

  • StateBackend interface (read/write/exists/list/remove/doctor)
  • OrphanBranchBackend - state in refs/heads/squad-state via git plumbing
  • FilesystemBackend - current behavior, fallback
  • 15 tests all passing including state survives branch switches

POC only - NOT wired into runtime. Proves the concept works.

Refs #678, #643, #498, #670

Testing

  • 15/15 tests pass
  • Purely additive - no existing code modified

Breaking Changes

None

Waivers

None

Proof of concept for storing Squad state in a git orphan branch instead
of the working tree. State stored this way is immune to branch switches,
survives gitignored .squad/, and never pollutes the working tree.

Implements:
- StateBackend interface (read/write/exists/list/remove/doctor)
- OrphanBranchBackend — state in refs/heads/squad-state orphan branch
- FilesystemBackend — state on disk (current behavior, fallback)
- 15 tests all passing, including:
  - State survives branch switches (the #643 scenario)
  - State survives with gitignored .squad/
  - Nested directory support (agents/fido/charter.md)
  - Doctor health checks

This is a POC — not wired into the runtime yet. See RFC in #678.

Refs #678, #643, #498, #670

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 29, 2026 13:36
@tamirdresher tamirdresher added do-not-merge PR is for discussion/review only - do not merge squad Squad triage inbox — Lead will assign to a member squad:eecom Assigned to EECOM (Core Dev) squad:flight Assigned to Flight (Lead) type:spike Research/investigation — produces a plan, not code labels Mar 29, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a proof-of-concept state persistence abstraction to the Squad SDK, exploring storing .squad/-style state inside a git orphan branch (squad-state) so it survives branch switches and avoids working-tree/gitignore fragility.

Changes:

  • Introduces a StateBackend interface + StateBackendHealth diagnostics contract.
  • Adds OrphanBranchBackend (git-plumbing-based persistence) and FilesystemBackend (current behavior/fallback).
  • Adds vitest coverage validating basic CRUD + “survives branch switches” behavior and doctor() health reporting.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
test/state-backend.test.ts New tests for OrphanBranchBackend + FilesystemBackend and backend health via doctor().
packages/squad-sdk/src/state/state-backend.ts Defines the StateBackend interface and health shape.
packages/squad-sdk/src/state/orphan-branch-backend.ts Implements git orphan-branch persistence using git show, commit-tree, and update-ref.
packages/squad-sdk/src/state/filesystem-backend.ts Implements on-disk persistence as a fallback backend.
packages/squad-sdk/src/state/index.ts Barrel exports for the new state module.

Comment on lines +38 to +43
git(['init'], repoDir);
git(['config', 'user.email', 'test@test.com'], repoDir);
git(['config', 'user.name', 'Test'], repoDir);
// Create an initial commit so we have a main branch
git(['commit', '--allow-empty', '-m', 'initial'], repoDir);
backend = new OrphanBranchBackend(repoDir);
Copy link

Copilot AI Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests assume the default branch is named main (e.g., later git checkout main), but git init may default to master depending on git config/version. To avoid flakes, initialize with git init -b main (or rename the initial branch via git branch -M main) before checking out main.

Copilot uses AI. Check for mistakes.
Comment on lines +139 to +154
it('state survives even with gitignored .squad/', async () => {
await backend.init();
await backend.write('team.md', '# Gitignored Scenario');

// Simulate the #643 scenario: .squad/ is gitignored
execFileSync('git', ['checkout', '-b', 'feature/gitignore-test'], {
cwd: repoDir, encoding: 'utf-8', stdio: 'pipe',
});
execFileSync('git', ['checkout', 'main'], {
cwd: repoDir, encoding: 'utf-8', stdio: 'pipe',
});

// State should survive because it's NOT in the working tree
const content = await backend.read('team.md');
expect(content).toBe('# Gitignored Scenario');
});
Copy link

Copilot AI Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test named "state survives even with gitignored .squad/" doesn’t actually create a .gitignore entry nor any untracked .squad/ files, so it isn’t exercising the #643 failure mode it references. Either simulate the real scenario (create .squad/ + gitignore + branch switch and assert the working-tree files would be removed while backend state persists) or rename the test to reflect what it actually validates (branch switching only).

Copilot uses AI. Check for mistakes.
}

async remove(path: string): Promise<void> {
const baseTree = this.git(['rev-parse', `${this.branch}^{tree}`]).trim();
Copy link

Copilot AI Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove() computes baseTree but never uses it. Dropping the unused variable will reduce noise and avoid suggesting there’s logic missing.

Suggested change
const baseTree = this.git(['rev-parse', `${this.branch}^{tree}`]).trim();

Copilot uses AI. Check for mistakes.
Comment on lines +37 to +44
// Create an empty orphan branch with an initial commit
const emptyTree = this.git(['hash-object', '-t', 'tree', '/dev/null']).trim()
|| this.git(['mktree'], '').trim(); // Windows fallback: empty stdin to mktree
const commitHash = this.git(
['commit-tree', emptyTree, '-m', 'Initialize squad-state branch']
).trim();
this.git(['update-ref', `refs/heads/${this.branch}`, commitHash]);
}
Copy link

Copilot AI Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

init() tries to compute an empty tree via git hash-object -t tree /dev/null and then fall back with || ..., but this.git() throws on failure so the fallback is never reached (and /dev/null won’t exist on Windows). Use a try/catch around the hash-object call or just use git mktree with empty stdin for a portable empty tree.

Copilot uses AI. Check for mistakes.
Comment on lines +91 to +107
async list(dir: string): Promise<string[]> {
try {
const output = this.git([
'ls-tree', '--name-only', this.branch,
]);
if (!output.trim()) return [];
const allFiles = output.split('\n').filter(Boolean);
if (dir === '.' || dir === '') {
return allFiles;
}
return allFiles
.filter(f => f.startsWith(`${dir}/`))
.map(f => f.slice(dir.length + 1));
} catch {
return [];
}
}
Copy link

Copilot AI Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

list(dir) currently runs git ls-tree --name-only <branch> which only returns top-level entries. That means list('agents') will return [] even if agents/fido/charter.md exists (unlike FilesystemBackend.list, which lists the directory’s contents). Consider using git ls-tree --name-only <branch>:<dir> for non-root directories (and <branch> for root) so dir works as intended.

Copilot uses AI. Check for mistakes.
Copilot review fixes:
1. Use 'git init -b main' for portable default branch name
2. Use 'git mktree' for empty tree (not /dev/null — Windows compat)
3. Remove unused baseTree variable in remove()
4. Fix list(dir) to use 'git ls-tree branch:dir' for subdirectories
5. Add real gitignore + .squad/ simulation to #643 test

Expanded test suite (15 → 36 tests) covering 7 scenarios:
- Scenario 1: Basic CRUD (9 tests)
- Scenario 2: Nested paths + subdirectory listing (5 tests)
- Scenario 3: Branch switch survival - #643 core fix (4 tests)
- Scenario 4: Edge cases - empty, unicode, large, many files (8 tests)
- Scenario 5: Doctor health checks (4 tests)
- Scenario 6: E2E full lifecycle simulation (2 tests)
- Scenario 7: FilesystemBackend comparison (4 tests)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tamirdresher
Copy link
Copy Markdown
Collaborator Author

Test Results — 36/36 Passing ✅

All 5 Copilot review findings addressed + test suite expanded from 15 → 36 tests.

Scenario Coverage

# Scenario Tests What It Proves
1 Basic CRUD 9 init, read, write, overwrite, exists, list, remove, idempotent init, working tree isolation
2 Nested Paths 5 single-level nesting, 3-level deep nesting, multiple files in same dir, sibling dirs, list('agents') subdirectory listing
3 Branch Switch Survival (#643) 4 checkout + back, 5 rapid switches, gitignored .squad/ with real .gitignore + working tree files, write-while-on-feature-branch
4 Edge Cases 8 empty content, unicode/emoji, 100KB large file, 20 files batch, markdown with code blocks, read-before-init, exists-before-init, list-before-init
5 Doctor Health 4 missing branch, healthy after init, file count reporting, non-git directory
6 E2E Lifecycle 2 full init→write team/routing/decisions/agents→branch switch→resume→write more→switch back cycle; verify orphan branch has proper git commit history
7 FilesystemBackend 4 read/write, nested dirs, healthy report, unhealthy for missing dir

Key Copilot Fixes Applied

  1. git init -b main — portable default branch name
  2. git mktree with empty stdin — no /dev/null (Windows compat)
  3. ✅ Removed unused baseTree in remove()
  4. list(dir) now uses git ls-tree branch:dir for subdirectories
  5. ✅ Gitignore test creates real .gitignore + .squad/ working tree files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge PR is for discussion/review only - do not merge squad:eecom Assigned to EECOM (Core Dev) squad:flight Assigned to Flight (Lead) squad Squad triage inbox — Lead will assign to a member type:spike Research/investigation — produces a plan, not code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants