Skip to content

Fork contributions: bug fixes, Windows compatibility, pipeline resilience, and new features #16

@joshbouncesecurity

Description

@joshbouncesecurity

I've been using OpenAnt (primarily on Windows) and have accumulated some changes in my fork that I'd like to contribute back. I've already opened PR #15 (test suite + CI). Before submitting the rest as individual PRs, I wanted to check which changes you'd be interested in.

I have working implementations for all of these — happy to open PRs for whichever ones you'd like. Check the ones you're interested in, or let me know if you have questions about any of them.

Testing & CI

  • 1. Test suite + CI (already open as PR test: add pytest test suite and CI workflow #15) — 60 pytest tests covering parsers, token tracking, language detection, and Go CLI integration. GitHub Actions CI on Linux, macOS, and Windows.
  • 2. Add ruff lint to CI — Adds ruff with two rules: F821 (undefined name) and F811 (redefined unused name). Unlike compiled languages, Python won't report an undefined name until that code path executes at runtime, so a missing import can ship undetected and only crash when a user hits that branch. These two rules catch that statically with zero false positives and no style noise.

Bug Fixes

These exist in the current codebase.

  • 3. Findings count shows 0 in build-outputPrintBuildOutputSummary in the Go CLI expects findings_count in the JSON response, but the Python CLI returns only {"pipeline_output_path": path} — so "Findings included: 0" is always displayed regardless of actual results.
  • 4. Parse defaults to --level all instead of reachable — The Go CLI parse command defaults --level to "all", while both scan and the Python CLI default to "reachable". This means openant parse standalone produces a different (larger, noisier) dataset than openant scan with no indication to the user.
  • 5. Analyze summary missing verdict categories — The Python analyzer tracks 6 verdict categories (vulnerable, bypassable, inconclusive, protected, safe, errors) but PrintAnalyzeSummary only displays Vulnerable and Safe — so the totals don't add up. PrintScanSummaryV2 already displays all categories correctly; analyze should match.
  • 6. Report paths missing from Go CLI outputPrintReportSummary expects html_path/csv_path/summary_path keys, but ReportResult.to_dict() only returns output_path and format — so "Reports Generated" is always blank.
  • 7. Incomplete call graph for TypeScript/NestJS codebases using dependency injection — The TypeScript parser doesn't extract constructor parameter types, so dependency-injected service calls (e.g., this.userService.findById()) are unresolved in the call graph. This means the security analysis silently misses data flow through injected services — a major blind spot for most production NestJS apps. This adds DI-aware resolution by extracting constructorDeps metadata from the AST and using it to resolve this.service.method() calls to the correct class.

Windows Compatibility

These prevent OpenAnt from working correctly on Windows.

  • 8. JS/Go parser path handling on Windowspath.relative() produces backslash paths on Windows, but ts-morph treats backslashes as escape characters — so the JS parser finds 0 files. Additionally, Windows \r\n line endings leave trailing \r in file lists, and Unicode symbols (✓✗→) crash on cp1252 consoles.
  • 9. UTF-8 file I/O across the codebase — All bare open() calls use the system encoding (cp1252 on Windows), causing charmap codec can't decode errors on any target codebase containing non-ASCII characters. This adds centralized UTF-8 helpers (open_utf8, read_json, write_json, run_utf8) and migrates all file I/O.

Pipeline Resilience

New capabilities addressing the cost and time implications of long-running scans failing mid-way.

  • 10. Crash recovery: checkpoint and resume — Enhance already supports checkpoint/resume via --checkpoint <path>, but it's opt-in and manual. This makes checkpointing always-on and automatic (replaces --checkpoint with --fresh to opt out), extends per-unit checkpointing to analyze and verify (which currently have none), adds scan step-level resume (re-running skips completed steps), adds --fresh to parse (so parser improvements take effect without manually deleting dataset.json), and wraps all JSON writes in atomic temp-file-then-rename to prevent corrupt files on crash. Currently, if a multi-hour scan crashes mid-analyze or mid-verify, all progress (and API spend) for that stage is lost.
  • 11. Auto-retry errored units — Enhance and analyze automatically retry units that errored on the previous run instead of treating errors as complete. --skip-errors flag to opt out. Currently, API errors (rate limits, timeouts) permanently mark units as "done" — the only recovery is to re-process everything.
  • 12. Parallel LLM callsThreadPoolExecutor-based parallelization for enhance, analyze, and verify with lock-protected checkpoints. --concurrency/-j flag (default 4). A scan that takes 2 hours serially completes in ~30 minutes with --concurrency 4.

New Features

  • 13. Auto-detect language in init — Makes --language optional by auto-detecting the project language from file extensions. Also makes git repository optional for local paths. Reduces friction for new users — openant init just works.
  • 14. Auto-detect dependency changes — Go CLI hashes pyproject.toml after pip install -e and re-runs install automatically when dependencies change. Prevents stale venv issues after git pull.
  • 15. Centralize model IDs — Single model_config.py with MODEL_PRIMARY/MODEL_AUXILIARY/MODEL_DEFAULT constants, replacing hardcoded model strings across 15 files. Makes model updates a one-line change instead of find-and-replace across the codebase.
  • 16. Migrate to Claude Agent SDK — Replaces the anthropic API backend with the Claude Agent SDK. The SDK handles both API key auth and local Claude Code session auth natively, and provides Read/Grep/Glob/Bash tools for enhance/verify — replacing the manual multi-turn tool loop. The verify stage in particular benefits significantly: the SDK's native tool use gives the verifier direct access to search and read the codebase, producing more accurate verdicts than the current custom tool dispatch. Also enables running OpenAnt without a separate API key for developers who have Claude Code.
  • 17. generate-context CLI command with auto-discovery — Adds openant generate-context [repo-path] as a standalone pipeline step to generate application_context.json. Fully integrated with the project system (openant init / project switch) — defaults output to the project scan directory. Also wires up auto-discovery of application_context.json in analyze and verify commands (both Go and Python CLIs) so --app-context is no longer required when the file exists in the scan dir. Previously, running individual steps required manually passing --app-context to every command or skipping application context entirely.
  • 18. Override merge mode for generate-context — When generate-context detects a manual override file (OPENANT.md/OPENANT.json), it now prompts the user to choose: use (as-is, skip LLM), merge (feed override into LLM alongside other sources), or ignore (skip override, generate from scratch). New --override-mode <use|merge|ignore> flag bypasses the prompt for CI/automation. --force kept as backward-compatible shortcut for --override-mode ignore. Previously, override files were all-or-nothing — either they fully replaced LLM generation or were ignored entirely, with no way to combine developer-provided hints with LLM analysis.


Detailed implementation notes, dependencies, and cherry-pick risks

Detailed Implementation Notes

Change 1: Test suite + CI (fork PRs #1 + #4, already upstream PR #15)

  • Dependencies: None — this is the foundation for all other changes
  • Cherry-pick difficulty: Already submitted

Change 2: Ruff lint in CI (fork PR #8, partial)

  • Files: .github/workflows/test.yaml
  • Dependencies: Change 1 (CI workflow to add the step to)
  • Cherry-pick difficulty: Clean
  • Note: Run ruff check . against upstream/master before submitting to verify clean baseline

Change 3: Findings count (fork PR #12)

  • Files: openant/cli.py (build-output handler), core/verifier.py (cached path)
  • Dependencies: None
  • Cherry-pick difficulty: Clean

Change 4: Parse --level default (fork PR #16)

  • Files: apps/openant-cli/cmd/parse.go
  • Dependencies: None
  • Cherry-pick difficulty: Clean (one-line change)

Change 5: Analyze summary verdicts (fork PR #18)

  • Files: apps/openant-cli/internal/output/formatter.go
  • Dependencies: None
  • Cherry-pick difficulty: Clean

Change 6: Report paths (fork PR #19)

  • Files: core/schemas.py (ReportResult), openant/cli.py (report handler)
  • Dependencies: None
  • Cherry-pick difficulty: Clean

Change 7: DI-aware call resolution for TypeScript/NestJS (fork PR #20)

  • Files: parsers/javascript/typescript_analyzer.js, utilities/agentic_enhancer/agent.py, utilities/agentic_enhancer/prompts.py
  • Dependencies: None (parser-level change)
  • Cherry-pick difficulty: Clean

Change 8: JS/Go parser Windows paths (fork PR #3)

  • Files: parsers/javascript/typescript_analyzer.js, parsers/javascript/test_pipeline.py, parsers/go/test_pipeline.py
  • Dependencies: None
  • Cherry-pick difficulty: Clean

Change 9: UTF-8 file I/O (fork PR #13)

  • Files: New utilities/file_io.py, plus migrations across ~20 files
  • Dependencies: None (but benefits from test suite for coverage)
  • Cherry-pick difficulty: Clean — large diff but mechanical replacement
  • Note: Cross-cutting change that touches many files. Best submitted early to avoid conflicts with other PRs.

Change 10: Checkpoint and resume (fork PRs #7, #9, #10, #21 — 3 stages + --fresh for parse)

Implemented as 3 sequential PRs plus a follow-up:

Stage Fork PR Scope
1 #7 Atomic writes (atomic_write_json) + enhance auto-checkpoint + --fresh flag
2 #9 Analyze/verify per-unit checkpointing + --fresh for analyze/verify
3 #10 Scan step-level resume (skip completed steps on re-run)
#21 --fresh flag for parse (forces full reparse, clears dataset in scan)
  • Dependencies: Stage 2 and 3 depend on Stage 1. Stage 2 and 3 are independent of each other. PR #21 depends on Stage 1 for --fresh pattern consistency.
  • Cherry-pick difficulty: Clean — primarily adds new code rather than modifying existing code
  • Recommendation: Submit as 3-4 sequential PRs to upstream
  • Important implementation details (bugs found and fixed during fork development):
    • In finding_verifier.py, the checkpoint must not add errored findings to completed_keys — otherwise errored findings are permanently marked as "done" and can never be retried on resume. Only add to completed_keys inside the try block after successful verification.
    • When analyze --verify --fresh chains both commands, the fresh flag must be forwarded to the run_verification() call, otherwise verify will skip with "Already complete" even though analyze just re-ran. (See fork PR #22 for the fix.)
    • The os module must be imported in context_enhancer.py for checkpoint cleanup (os.path.exists(), os.remove()) — the upstream file doesn't currently use os, but the checkpoint code will need it.

Change 11: Auto-retry errored units (fork PR #15)

  • Dependencies: Checkpoint/resume (change 10)
  • Cherry-pick difficulty: Clean

Change 12: Parallel LLM calls (fork PR #17)

  • Dependencies: Checkpoint/resume (change 10), auto-retry (change 11)
  • Cherry-pick difficulty: Moderate — touches llm_client.py which diverged in the fork. The new parallel_executor.py module and stage-level changes are clean, but thread-safety changes to TokenTracker and load_dotenv movement need to target upstream's version of llm_client.py.

Change 13: Auto-detect language (fork PR #6)

  • Dependencies: None
  • Cherry-pick difficulty: Clean

Change 14: Auto-detect dependency changes (fork PR #23)

  • Dependencies: None (Go CLI only)
  • Cherry-pick difficulty: Clean

Change 15: Centralize model IDs (fork PR #24)

  • Dependencies: None
  • Cherry-pick difficulty: Clean
  • Note: Upstream may want different default model IDs but the centralization pattern is valuable

Change 16: Claude Agent SDK migration (fork PR #25)

This is the largest change. It subsumes all earlier local Claude attempts (fork PRs #2, #5, #11, #14) which should NOT be submitted individually.

Key benefits:

  • Single backend: All LLM calls route through the SDK

  • Local Claude Code support: SDK natively supports both API key auth and local session auth

  • Native tools: SDK provides Read/Grep/Glob/Bash tools natively, replacing the manual tool loop

  • Accurate cost tracking: Uses SDK's ResultMessage.total_cost_usd

  • Simpler dependency: Replaces anthropic with claude-agent-sdk in pyproject.toml

  • Dependencies: Centralize model IDs (change 15) should go first. Concurrency (change 12) should go before or be merged with this.

  • Cherry-pick difficulty: Cannot be cherry-picked — must be re-authored as a standalone diff against upstream (see risks section below)

Change 17: generate-context CLI command with auto-discovery (fork PR #26)

  • Files: New apps/openant-cli/cmd/generatecontext.go, modified apps/openant-cli/cmd/root.go, apps/openant-cli/cmd/analyze.go, apps/openant-cli/cmd/verify.go, libs/openant-core/openant/cli.py, libs/openant-core/tests/test_go_cli.py
  • Documentation: Also includes doc updates (fork PR #28) to PIPELINE_MANUAL.md, CURRENT_IMPLEMENTATION.md, README.md, and DOCUMENTATION.md — these should be included when cherry-picking this change.
  • Dependencies: None — standalone addition reusing existing context.application_context module
  • Cherry-pick difficulty: Clean
  • Testing status: Automated tests cover help output and API key validation. Manual testing with an API key is still needed for: generate-context with active project, auto-discovery in analyze/verify, --force/--show-prompt/--json flags, and explicit --app-context precedence over auto-discovery.

Change 18: Override merge mode for generate-context (fork PR #27)

  • Files: Modified apps/openant-cli/cmd/generatecontext.go, libs/openant-core/context/application_context.py, libs/openant-core/openant/cli.py, libs/openant-core/tests/test_go_cli.py, plus docs (CLAUDE.md, CURRENT_IMPLEMENTATION.md, PIPELINE_MANUAL.md, context/OPENANT_TEMPLATE.md)
  • Dependencies: Change 17 (generate-context command must exist)
  • Cherry-pick difficulty: Clean — builds on top of change 17's generatecontext.go and application_context.py
  • Implementation: Go CLI adds interactive prompt (following uninstall.go pattern) and --override-mode flag. Python core adds find_override_file() helper, override_mode parameter to generate_application_context(), and merge prompt supplement fed to the LLM. Terminal detection via os.Stdin.Stat() skips the prompt in non-interactive/CI environments (defaults to use).
  • Testing status: Automated tests cover --override-mode in help output and --force/--override-mode mutual exclusion. Manual testing needed for: interactive prompt with a repo containing OPENANT.md, merge mode LLM output (verify with --show-prompt), and non-interactive default behavior.

Superseded Changes (Not for Upstream)

These were intermediate steps toward local Claude Code support, fully superseded by the SDK migration (change 16). They should NOT be submitted upstream individually.

Fork PR Description Why superseded
#2 feat: support local Claude Code session for LLM calls Replaced by SDK's native auth support in #25
#5 test: add tests for local Claude Code mode Tests for LocalClaudeClient which is deleted in #25
#8 fix: missing os import in context_enhancer Bug introduced by fork's checkpoint code, not present in upstream
#11 feat: add text-based tool use simulation to LocalClaudeClient Replaced by SDK's native tool support in #25
#14 fix: pass prompt via stdin to avoid WinError 206 Fix for local_claude.py which is deleted in #25
#22 fix: pass --fresh flag to chained verify Depends on --fresh which is introduced by change 10

Recommended Submission Order

The order below respects dependencies and minimizes merge conflicts:

# Change Fork PR(s) Type Depends on
1 1 #1, #4 Tests + CI — (already upstream PR #15)
2 2 #8 (partial) Ruff lint in CI 1
3 8 #3 Windows JS/Go parser paths
4 9 #13 UTF-8 file I/O 1 (for tests)
5 3 #12 Findings count bug fix
6 4 #16 Parse --level default
7 5 #18 Analyze summary verdicts
8 6 #19 Report paths
9 7 #20 DI-aware call resolution
10 13 #6 Auto-detect language
11 10 #7, #9, #10, #21 Checkpoint, resume, and --fresh (3-4 PRs)
12 11 #15 Auto-retry errored units 10
13 12 #17 Parallel LLM calls 10, 11
14 14 #23 Auto-detect dependency changes
15 15 #24 Centralize model IDs
16 16 #25 Claude Agent SDK migration 15, 12
17 17 #26 generate-context CLI command + auto-discovery
18 18 #27 Override merge mode for generate-context 17

Cherry-Pick Risks & Notes

Critical: Each PR must be rebased, not cherry-picked directly

Every fork PR was developed incrementally on top of previous fork PRs (including the superseded local Claude ones: #2, #5, #11, #14). Direct git cherry-pick will produce diffs relative to the fork's history, not upstream's. Each change must be rebased or re-authored as a standalone diff against upstream/master (or against the upstream branch that includes previously merged contributions).

Import chain dependencies

Several fork PRs create new modules that later PRs import. Missing a dependency in the chain will cause ImportError at runtime:

Module Created by Required by
core/utils.py (atomic_write_json) Change 10 (resume) Changes 10-12, 9
utilities/file_io.py (open_utf8, etc.) Change 9 (UTF-8 I/O) Every subsequent PR that touches file I/O
utilities/parallel_executor.py (run_parallel) Change 12 (concurrency) — (leaf)
utilities/model_config.py (MODEL_PRIMARY, etc.) Change 15 (model IDs) Change 16 (SDK)

cli.py flag accumulation

The fork incrementally added CLI flags across many PRs (--fresh, --skip-errors, --concurrency). Each PR's diff assumes the previous flags already exist in cli.py. When rebasing against upstream, each PR needs to add its flags against upstream's version of cli.py, not the fork's. The same applies to the Go CLI cmd/*.go files — multiple PRs add Cobra flags to the same files.

Go CLI cmd files

Upstream has 17 files in apps/openant-cli/cmd/ — more than the fork touches. Upstream may have modified the same cmd files the fork changed. Each Go CLI PR should be diffed against upstream's current version of those files.

llm_client.py is the highest-conflict file

This file diverged the most between fork and upstream:

  • Upstream: Uses anthropic.Anthropic() directly, model IDs hardcoded as "claude-opus-4-20250514" / "claude-sonnet-4-20250514"
  • Change 12 (concurrency): Added threading.Lock to TokenTracker, moved load_dotenv() to module-level
  • Change 15 (model IDs): Replaced hardcoded strings with model_config imports
  • Change 16 (SDK): Rewrote entirely

Changes 12, 15, and 16 each need to be re-authored against upstream's version of this file.

pyproject.toml dependency divergence

Upstream has anthropic>=0.40.0. The fork replaced this with claude-agent-sdk>=0.1.48 in change 16. All changes before 16 should leave pyproject.toml unchanged. Change 16 is the only one that swaps the dependency.

Ruff lint may fail on upstream code

Change 2 adds ruff with F821/F811 rules to CI. If upstream has introduced code that violates these rules, CI will fail. Run ruff check . against upstream/master before submitting to verify clean baseline.

context_enhancer.py and finding_verifier.py evolution

These files were modified by multiple fork PRs:

  • context_enhancer.py: change 9 (UTF-8 I/O), change 12 (concurrency), change 16 (SDK)
  • finding_verifier.py: change 10 (checkpointing), change 12 (concurrency), change 16 (SDK rewrite)

Later changes (especially 12 and 16) need manual attention when rebasing.

progress.py modified by two independent features

  • Change 10 (resume): Adds completed offset parameter to ProgressReporter.__init__
  • Change 12 (concurrency): Adds threading.Lock to ProgressReporter.report()

If submitted as separate PRs, the second must account for the first.

SDK migration (change 16) cannot be cherry-picked at all

Change 16 in the fork is a diff that deletes local_claude.py and removes LocalClaudeClient references — neither of which exist in upstream. Against upstream, it needs to be a fresh rewrite that replaces anthropic usage with claude-agent-sdk in upstream's current files. This is effectively a new PR authored against upstream, informed by the fork's implementation.

Upstream may have moved

All analysis is based on upstream/master at time of writing (2026-03-23). If upstream merges other PRs before these contributions land, additional conflicts may arise. Re-fetch and recheck before each submission.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions