Skip to content

feat: tree-sitter symbol indexing via jcodemunch#135

Merged
github-actions[bot] merged 5 commits intomasterfrom
feature/symbol-index-service
Mar 6, 2026
Merged

feat: tree-sitter symbol indexing via jcodemunch#135
github-actions[bot] merged 5 commits intomasterfrom
feature/symbol-index-service

Conversation

@jordanpartridge
Copy link
Copy Markdown
Contributor

@jordanpartridge jordanpartridge commented Mar 6, 2026

Summary

  • New SymbolIndexService — reads jcodemunch's JSON index format, does byte-offset fseek/fread symbol retrieval natively in PHP (O(1), no re-parsing)
  • Updated index-code command — tree-sitter AST parsing via jcodemunch, with incremental indexing (SHA-256 hash-based change detection)
  • Updated search-code command — weighted symbol search (name/signature/docstring/keyword scoring), file outlines, kind/file filters, --show-source via byte offsets

What this replaces

The old CodeIndexerService used regex function extraction and 2000-char chunking into Qdrant embeddings. The new approach extracts 527 discrete symbols from 97 PHP files using tree-sitter AST parsing — with exact byte offsets for instant source retrieval, no embedding cost for code navigation.

Test plan

  • 37 unit tests for SymbolIndexService (search, retrieval, outline, change detection, path traversal, version handling)
  • 9 feature tests for IndexCodeCommand (index, incremental, list repos, error paths)
  • 15 feature tests for SearchCodeCommand (search, filters, outline, source display)
  • 100% coverage on all new/modified files
  • PHPStan level 8 clean
  • Full suite: 1145 passed, 0 failures

Summary by CodeRabbit

  • New Features

    • Symbol-based code indexing and search with filtering by kind, file pattern, and repository.
    • File outline functionality for navigating code structure.
  • Changes

    • Renamed "Odin Sync" to "Remote Sync" throughout the application and configuration.
    • Updated code indexing from file-based to symbol-based approach with incremental support.
    • Modified code search to operate on indexed symbols rather than raw files.
  • Documentation

    • Updated architecture diagrams, configuration examples, and command references to reflect Remote Sync terminology.

jordanpartridge and others added 5 commits February 11, 2026 10:49
Depersonalizes the codebase by replacing all "Odin" references with
generic "Remote" terminology. The remote sync feature is optional
infrastructure for home labs and teams sharing a centralized vector
store — naming should reflect that, not a specific server.

Renames: OdinSyncService → RemoteSyncService, OdinSyncCommand →
RemoteSyncCommand, sync:odin → sync:remote, --odin → --remote,
ODIN_* env vars → REMOTE_SYNC_*, docker-compose.odin.yml →
docker-compose.remote.yml. All tests updated accordingly.
These are already in .gitignore but were previously committed.
No code references these views — dead code in a CLI app.
Replace regex-based code chunking with AST-aware symbol extraction
using jcodemunch's tree-sitter parser. The new SymbolIndexService reads
jcodemunch's JSON index format and does byte-offset retrieval natively
in PHP, enabling O(1) symbol source lookup without re-parsing.

- Add SymbolIndexService with search, outline, and change detection
- Update IndexCodeCommand to use tree-sitter indexing with incremental support
- Update SearchCodeCommand with symbol-first search, kind/file filters, and file outlines
- 100% test coverage on all new/modified files (61 tests, 134 assertions)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 6, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e772eff8-3d15-4c52-bd27-b2b49b0588d6

📥 Commits

Reviewing files that changed from the base of the PR and between ba214dc and af980dc.

📒 Files selected for processing (40)
  • .env.example
  • CLAUDE.md
  • MISSION.md
  • README.md
  • ROADMAP.md
  • app/Commands/IndexCodeCommand.php
  • app/Commands/KnowledgeStatsCommand.php
  • app/Commands/RemoteSyncCommand.php
  • app/Commands/SearchCodeCommand.php
  • app/Commands/Service/DownCommand.php
  • app/Commands/Service/LogsCommand.php
  • app/Commands/Service/StatusCommand.php
  • app/Commands/Service/UpCommand.php
  • app/Providers/AppServiceProvider.php
  • app/Services/RemoteSyncService.php
  • app/Services/SymbolIndexService.php
  • app/Services/ThemeClassifierService.php
  • config/services.php
  • docker-compose.remote.yml
  • resources/views/site/categories.blade.php
  • resources/views/site/entry.blade.php
  • resources/views/site/index.blade.php
  • resources/views/site/layout.blade.php
  • resources/views/site/tags.blade.php
  • storage/framework/views/0f479c632229ee0a8ced4fa17a2fd1cc.php
  • storage/framework/views/26dd2becec94e79077a33621a63a7b11.php
  • storage/framework/views/bb5d12f3b3a26511ea3fac7651fa7849.php
  • storage/framework/views/e5aa6538698ba9e145c6fdac9ec1eb3d.php
  • storage/framework/views/e6d56113987c2dd4afd52a2de9a8da50.php
  • tests/Feature/Commands/IndexCodeCommandTest.php
  • tests/Feature/Commands/RemoteSyncCommandTest.php
  • tests/Feature/Commands/SearchCodeCommandTest.php
  • tests/Feature/Commands/Service/DownCommandTest.php
  • tests/Feature/Commands/Service/LogsCommandTest.php
  • tests/Feature/Commands/Service/StatusCommandTest.php
  • tests/Feature/Commands/Service/UpCommandTest.php
  • tests/Feature/KnowledgeStatsCommandTest.php
  • tests/Unit/Services/RemoteSyncServiceTest.php
  • tests/Unit/Services/SymbolIndexServiceTest.php
  • tests/Unit/Services/ThemeClassifierServiceTest.php

📝 Walkthrough

Walkthrough

This pull request refactors the centralized sync system from "Odin" to a generic "Remote" nomenclature across configuration, services, and CLI commands. Additionally, it introduces a new SymbolIndexService for tree-sitter-based code symbol indexing and replaces the legacy CodeIndexerService throughout the codebase. Static site generation Blade templates are removed entirely.

Changes

Cohort / File(s) Summary
Configuration & Environment
.env.example, config/services.php, docker-compose.remote.yml
Renamed Odin sync environment variables and config keys to Remote equivalents; updated docker-compose example reference from docker-compose.odin.yml to docker-compose.remote.yml.
Documentation
CLAUDE.md, MISSION.md, README.md, ROADMAP.md
Updated service descriptions, architecture diagrams, and command references to reflect Remote Sync terminology instead of Odin; replaced sync:odin with sync:remote.
Sync Service Refactoring
app/Services/RemoteSyncService.php, app/Providers/AppServiceProvider.php, app/Commands/RemoteSyncCommand.php, app/Commands/KnowledgeStatsCommand.php
Renamed OdinSyncService to RemoteSyncService; updated method pullFromOdin() to pullFromRemote(); refactored all configuration reads and command invocations to use Remote equivalents; updated CLI output labels and status checks.
Symbol Indexing Service
app/Services/SymbolIndexService.php
Introduced new service providing tree-sitter-based symbol indexing with methods for indexing folders, searching symbols, retrieving source code, building file outlines, detecting changes, and listing repositories; supports incremental indexing and fuzzy symbol scoring.
Index & Search Commands
app/Commands/IndexCodeCommand.php, app/Commands/SearchCodeCommand.php
Replaced CodeIndexerService with SymbolIndexService; refactored IndexCodeCommand to use indexFolder() with incremental support and repository listing; updated SearchCodeCommand to use searchSymbols() with kind/file filtering and added --outline option for file structure visualization.
Service Commands
app/Commands/Service/UpCommand.php, app/Commands/Service/DownCommand.php, app/Commands/Service/LogsCommand.php, app/Commands/Service/StatusCommand.php
Replaced --odin flag with --remote; updated docker-compose file selection from docker-compose.odin.yml to docker-compose.remote.yml; changed environment label from "Odin (Remote)" to "Remote".
Theme Classifier
app/Services/ThemeClassifierService.php
Removed "odin" keyword from integrated-infrastructure theme keywords list.
Removed UI Templates
resources/views/site/categories.blade.php, resources/views/site/entry.blade.php, resources/views/site/index.blade.php, resources/views/site/layout.blade.php, resources/views/site/tags.blade.php, storage/framework/views/*
Deleted static site generation Blade templates including layout scaffold, index/entry/category/tag pages, and compiled view cache files; eliminates server-rendered knowledge base UI.
Test Updates — Sync Services
tests/Feature/Commands/RemoteSyncCommandTest.php, tests/Feature/KnowledgeStatsCommandTest.php, tests/Unit/Services/RemoteSyncServiceTest.php
Updated tests to mock RemoteSyncService instead of OdinSyncService; replaced sync:odin command invocation with sync:remote; updated configuration key references from services.odin.* to services.remote.*; adjusted assertions for "Remote" terminology.
Test Updates — Indexing & Search
tests/Feature/Commands/IndexCodeCommandTest.php, tests/Feature/Commands/SearchCodeCommandTest.php, tests/Unit/Services/SymbolIndexServiceTest.php
Replaced CodeIndexerService mocks with SymbolIndexService; updated test expectations for indexFolder() and searchSymbols() method signatures; added comprehensive test coverage for symbol search, source retrieval, file outlining, change detection, and repository listing; added helper createSymbolResult() for test data generation.
Test Updates — Service Commands
tests/Feature/Commands/Service/UpCommandTest.php, tests/Feature/Commands/Service/DownCommandTest.php, tests/Feature/Commands/Service/LogsCommandTest.php, tests/Feature/Commands/Service/StatusCommandTest.php
Updated test invocations and assertions to use --remote flag instead of --odin; changed expected docker-compose file from docker-compose.odin.yml to docker-compose.remote.yml.
Theme Tests
tests/Unit/Services/ThemeClassifierServiceTest.php
Updated test data tag from "odin" to "server" for Docker Deployment Configuration theme.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant SymbolIndexService
    participant Python as Python Indexer
    participant FileSystem as File System
    participant Index as Index Storage

    rect rgba(100, 200, 150, 0.5)
    Note over Client,Index: Indexing Flow
    Client->>SymbolIndexService: indexFolder(path, incremental)
    SymbolIndexService->>FileSystem: Validate path
    SymbolIndexService->>Python: Execute indexer subprocess
    Python->>FileSystem: Parse files with tree-sitter
    Python->>Index: Write symbol index (JSON)
    Index-->>Python: Confirm write
    Python-->>SymbolIndexService: Return results (JSON)
    SymbolIndexService-->>Client: Return success/failure payload
    end

    rect rgba(150, 150, 200, 0.5)
    Note over Client,Index: Search Flow
    Client->>SymbolIndexService: searchSymbols(query, repo, kind, pattern)
    SymbolIndexService->>Index: loadIndex(repo)
    Index-->>SymbolIndexService: Symbol metadata array
    SymbolIndexService->>SymbolIndexService: Filter by kind/pattern
    SymbolIndexService->>SymbolIndexService: Score symbols (fuzzy match)
    SymbolIndexService-->>Client: Return top results sorted by score
    end

    rect rgba(200, 150, 150, 0.5)
    Note over Client,Index: Source Retrieval
    Client->>SymbolIndexService: getSymbolSource(symbolId, repo)
    SymbolIndexService->>Index: findSymbol(symbolId)
    Index-->>SymbolIndexService: Symbol with byte offsets
    SymbolIndexService->>FileSystem: Read content via offset
    FileSystem-->>SymbolIndexService: Source code snippet
    SymbolIndexService-->>Client: Return source text
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Poem

🐰 A remote sync hops into place,
Symbols indexed at tree-sitter's pace,
Templates fade like old carrot crumbs,
New search flows bring knowledge that hums,
From Odin to Remote, the rabbit now runs! 🥬

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feature/symbol-index-service

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 6, 2026

📊 Coverage Report

Metric Coverage Threshold Status
Lines 97.9% 95%

Files Below Threshold

File Coverage Uncovered Lines
app/Enums/ObservationType.php 0% None
app/Exceptions/Qdrant/QdrantException.php 0% None
app/Integrations/Qdrant/Requests/ScrollPoints.php 0% None
app/Mcp/Servers/KnowledgeServer.php 0% None
app/Services/AgentHealthService.php 0% None
app/Mcp/Tools/RememberTool.php 66.7% 106, 107, 108, 109, 110... (+19 more)
app/Mcp/Tools/CorrectTool.php 70.4% 53, 54, 55, 56, 57... (+3 more)
app/Providers/McpServiceProvider.php 73.7% 44, 46, 48, 49, 50
app/Mcp/Tools/RecallTool.php 75.9% 81, 82, 83, 84, 85... (+15 more)
app/Commands/Concerns/ResolvesProject.php 80% 26

🏆 Synapse Sentinel Gate

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 6, 2026

🏆 Sentinel Certified

Tests & Coverage: 0 tests passed
Security Audit: No security vulnerabilities found
Pest Syntax: All test files use describe/it syntax


Add this badge to your README:

[![Sentinel Certified](https://img.shields.io/github/actions/workflow/status/conduit-ui/knowledge/gate.yml?label=Sentinel%20Certified&style=flat-square)](https://github.com/conduit-ui/knowledge/actions/workflows/gate.yml)

@github-actions github-actions bot merged commit 0d25c79 into master Mar 6, 2026
1 of 2 checks passed
@github-actions github-actions bot deleted the feature/symbol-index-service branch March 6, 2026 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant