Add automated perturbation benchmark for any paper

[r-uben]

Summary

Generalizes the seeded-perturbation benchmark (from PR #34) into a reusable module that works on any paper. Addresses @chenhaot's comment asking to "add more papers and make this seeded error part more general."

• Two-stage pipeline: (1) deterministic regex extraction identifies candidate spans (equations, numeric values, definitions, assumptions, claims, cross-references, conditions), then LLM picks minimal edits; (2) free-form LLM proposals validated by fuzzy match against the paper text
• Six error categories: numeric_parameter, operator_or_sign, symbol_binding, index_or_subscript, condition_or_assumption, claim_strengthening
• Validation layer: overlap detection, truncation checks (unbalanced delimiters), boundary garbage prevention
• Scoring: fuzzy + LLM-based matching of reviewer comments against injected errors
• Client fixes: OPENAI_BASE_URL for custom endpoints (EU, Azure); max_completion_tokens for GPT-5+/o-series models

CLI Usage

# Generate perturbations for any paper
openaireview perturb paper.pdf --model gpt-4.1 --output-dir ./results

# Review the corrupted paper
openaireview review results/paper_corrupted.md --method zero_shot

# Score the review against ground truth
openaireview score results/paper_perturbations.json results/paper_review.json

Initial Results (Nakamura & Steinsson 2018, QJE)

20 perturbations injected across 6 categories:

Review Method	Model	Comments	Recall	FP Rate
zero_shot	gpt-4.1	9	0%	100%
zero_shot	Claude Sonnet 4	4	0%	100%
consistency_check	gpt-4.1	11	25–30%	45–55%

The zero_shot prompt produces generic peer-review comments (ambiguity, exposition) but misses every injected error. A targeted consistency-checking prompt catches numeric, sign, and direction errors — demonstrating that the benchmark works and the current method has a blind spot for verifiable internal inconsistencies.

Test plan

• Pipeline runs end-to-end on Nakamura & Steinsson (2018)
• Scoring correctly matches reviewer comments to perturbations (fuzzy + LLM)
• Validates against truncated spans, overlapping edits, boundary collisions
• Client works with OpenAI EU endpoint (OPENAI_BASE_URL)
• Client works with GPT-5+ models (max_completion_tokens)
• Test on additional papers (neuroscience paper in progress)
• Add unit tests for extract/validate/inject