Skip to content

Smart Indexer#4

Open
danilin-em wants to merge 1 commit intomasterfrom
feature-smart-indexer
Open

Smart Indexer#4
danilin-em wants to merge 1 commit intomasterfrom
feature-smart-indexer

Conversation

@danilin-em
Copy link
Copy Markdown
Owner

No description provided.

Copilot AI review requested due to automatic review settings March 21, 2026 13:31
@danilin-em danilin-em changed the title enriched_overview_analysis Smart Indexer Mar 21, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an “enriched overview” generation path to VedCode by enhancing the file-tree output (directory metadata + configurable file depth) and introducing a new prompt + indexing stage that uses directory and root-file semantic summaries.

Changes:

  • Extend walker tree output to include per-directory aggregate metadata (extension counts + total size) and add TreeFileDepth to limit how deep file leaves are shown.
  • Add a new built-in prompt (DefaultEnrichedOverviewAnalysis) and wire it through config.
  • Add Stage 3 in the indexer to generate an enriched project overview using the tree + top-level directory summaries + root-level file summaries.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
internal/walker/walker.go Adds TreeFileDepth support and directory metadata aggregation/formatting in the rendered tree
internal/walker/walker_test.go Updates expected tree output and adds coverage for depth-based file rendering
internal/prompts/prompts.go Introduces DefaultEnrichedOverviewAnalysis prompt template
internal/indexer/indexer.go Adds Stage 3 enriched overview generation and helpers for top-level dir summaries
internal/config/config.go Adds YAML config for tree_file_depth and enriched_overview_analysis, sets defaults/merge rules
internal/config/config_test.go Adds test assertions for default/custom/home-merge of the new enriched prompt

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +238 to +240
if cfg.Indexer.TreeFileDepth <= 0 {
cfg.Indexer.TreeFileDepth = DefaultTreeFileDepth
}
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TreeFileDepth=0 is documented in walker.Options as a meaningful value ("root only"), but config loading treats 0 as "unset" (merge only copies when != 0, and defaults force <=0 to DefaultTreeFileDepth). This makes it impossible to configure root-only file display via YAML. Consider using a pointer (e.g., *int) to distinguish unset vs 0, or adopting a sentinel like -1 for "unset" and validating negative values.

Copilot uses AI. Check for mistakes.
logger.Info("\n--- Stage 3: Enriched project overview ---")

topLevelDirs := tracker.topLevelSummaries()
if len(topLevelDirs) > 0 || len(rootFileSummaries) > 0 {
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rootFileSummaries is built from concurrent file goroutines, but buildFilesSummariesText preserves slice iteration order. That order is nondeterministic, so the enriched overview prompt can change between runs even when inputs are identical. Sort rootFileSummaries by filePath (or sort inside buildFilesSummariesText) before rendering the prompt.

Suggested change
if len(topLevelDirs) > 0 || len(rootFileSummaries) > 0 {
if len(topLevelDirs) > 0 || len(rootFileSummaries) > 0 {
if len(rootFileSummaries) > 0 {
sort.Slice(rootFileSummaries, func(i, j int) bool {
return rootFileSummaries[i].FilePath < rootFileSummaries[j].FilePath
})
}

Copilot uses AI. Check for mistakes.
Comment on lines +692 to +695
if err := os.WriteFile(overviewPath, []byte(enrichedOverview), 0o644); err != nil {
logger.Error(fmt.Sprintf("Error saving enriched overview: %v", err))
} else {
logger.Info(fmt.Sprintf("Enriched project overview saved to %s", overviewPath))
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stage 3 writes the enriched overview back to overviewPath (project_overview.md), overwriting the Stage 1 structure-only overview saved earlier in the run. If both are useful, consider saving the enriched output to a separate file name (or appending a clearly separated section) to avoid losing the original output.

Suggested change
if err := os.WriteFile(overviewPath, []byte(enrichedOverview), 0o644); err != nil {
logger.Error(fmt.Sprintf("Error saving enriched overview: %v", err))
} else {
logger.Info(fmt.Sprintf("Enriched project overview saved to %s", overviewPath))
// Save enriched overview to a separate file to avoid overwriting the structure-only overview.
enrichedOverviewPath := overviewPath
if ext := filepath.Ext(overviewPath); ext != "" {
base := strings.TrimSuffix(overviewPath, ext)
enrichedOverviewPath = base + "_enriched" + ext
} else {
enrichedOverviewPath = overviewPath + "_enriched"
}
if err := os.WriteFile(enrichedOverviewPath, []byte(enrichedOverview), 0o644); err != nil {
logger.Error(fmt.Sprintf("Error saving enriched overview: %v", err))
} else {
logger.Info(fmt.Sprintf("Enriched project overview saved to %s", enrichedOverviewPath))

Copilot uses AI. Check for mistakes.
Comment on lines +382 to +386
entries := make([]extEntry, 0, len(n.extCounts))
for ext, count := range n.extCounts {
entries = append(entries, extEntry{ext, count})
}
sort.Slice(entries, func(i, j int) bool {
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

formatDirMeta renders every distinct extension entry in extCounts. In repositories with many file types this can make the tree (and thus LLM prompt) very large. Consider limiting the number of displayed extensions (e.g., top N + "other"), or gating metadata to shallow depths to keep prompt size predictable.

Copilot uses AI. Check for mistakes.
Comment on lines +307 to +309
if ext == "" {
ext = part // extensionless files: Makefile, Dockerfile
}
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New behavior for extensionless files sets ext to the full filename (e.g., "Makefile"), which affects directory metadata aggregation and ordering. There are no tests covering extensionless files or multi-extension directories; adding a focused test would help prevent regressions in extCounts formatting.

Suggested change
if ext == "" {
ext = part // extensionless files: Makefile, Dockerfile
}

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants