[Testing] Test Suite Overhaul - Epic

[itomek-amd]

Overview

Comprehensive test suite improvement using parallel development strategy. Build new tests_v2/ directory with improved structure while keeping existing tests/ intact, then cut over when complete.

Strategy

Parallel Development with Clean Cutover:

1. Keep tests/ intact - existing tests continue running in CI
2. Create tests_v2/ - new directory with improved structure
3. Incremental PRs - each sub-issue adds to tests_v2/
4. Final cutover PR - switch CI, remove old tests/, rename tests_v2/ → tests/

Benefits

• Zero disruption to existing CI
• Small, reviewable PRs
• Safe rollback if issues arise
• Clear before/after comparison

Sub-Issues

• [Testing] CLI Command Test Coverage + tests_v2 Foundation #239 - CLI Command Test Coverage + tests_v2 Foundation
• [Testing] Import Manifest System #240 - Import Manifest System
• [Testing] Core SDK Tests Migration #241 - Core SDK Tests Migration
• [Testing] Agent Tests Migration #242 - Agent Tests Migration
• [Testing] MCP Tests Migration #243 - MCP Tests Migration
• [Testing] Documentation Code Validation #244 - Documentation Code Validation
• [Testing] Final Verification Before Cutover #245 - Final Verification Before Cutover
• [Testing] CI Cutover - Switch to New Test Structure #246 - CI Cutover (FINAL)

Acceptance Criteria

• All sub-issues completed
• tests_v2/ has equivalent or better coverage than tests/
• All 218+ existing test behaviors preserved
• CI successfully switched to new test structure
• Coverage threshold ≥15%
• Anti-patterns fixed:
  • 13 _TOOL_REGISTRY.clear() calls removed
  • Hardcoded ports replaced with dynamic allocation
  • Proper pytest fixtures implemented

Related

• Documentation Review Tracking #173 - Documentation Review Tracking
• [Bug] Chat command doesn't find indexed documents + documentation gaps #207 - Chat/RAG bug fix

Plan Document

Full implementation plan: docs/plans/testing.mdx