Rewrite built-in fixtures in Python, collapse NormalizedFixture to a struct#645
Merged
MatthewMckee4 merged 19 commits intomainfrom Apr 11, 2026
Merged
Rewrite built-in fixtures in Python, collapse NormalizedFixture to a struct#645MatthewMckee4 merged 19 commits intomainfrom
MatthewMckee4 merged 19 commits intomainfrom
Conversation
MatthewMckee4
added
internal
Merging this PR will not alter performance
|
…struct
Built-in fixtures (`capsys`, `capfd`, `capsysbinary`, `capfdbinary`, `caplog`,
`tmp_path`, `tmpdir`, `tmp_path_factory`, `tmpdir_factory`, `monkeypatch`,
`recwarn`) were previously implemented as Rust `#[pyclass]` structs totalling
~1,800 lines. They are now plain Python generator functions in
`python/karva/_builtins.py`, discovered at runtime via `py.import("karva._builtins")`
and treated identically to user-defined fixtures.
This also collapses `NormalizedFixture` from a two-variant enum
(`BuiltIn` / `UserDefined`) to a plain struct, removing the branching
in `package_runner.rs` and `fixture_resolver.rs` that distinguished them.
`MockEnv` moves from a Rust `#[pyclass]` to a pure-Python class and is
now imported from `karva._builtins` instead of `karva._karva`.
Adds ty as a local pre-commit hook and configures it via pyproject.toml (python-version = "3.11", src includes python/). Fixes all diagnostics: suppresses unresolvable optional imports (py.path), adds SnapshotMismatchError to the _karva stub, and resolves property-name shadowing in ExceptionInfo.type and _WarningsChecker.list. Drops the _pytest._py.path fallback since karva has no pytest dependency.
Removes the py.path.local dependency from the tmpdir fixtures entirely — karva has no pytest compatibility requirement, so pathlib.Path is the right return type. Deletes _get_local_path_class() and updates the integration tests accordingly.
Moves all local classes to module level with complete type annotations on every method, parameter, and instance variable. Removes all # type: ignore and # ty: ignore suppression comments by fixing the underlying issues: cast(TextIO, ...) for binary stream assignment, _WarningList TypeAlias to avoid .list property shadowing the builtin, TypeVars for setitem/delitem public API, and Generator return types on all fixture functions. _CapLogHandler now owns its records list and _CapLog accesses them through the handler directly.
Functions prefixed with _ in karva._builtins are internal helpers, not fixtures. Skipping them prevents noisy debug log lines when running with -vv, and updates the stale snapshots.
uvx is not available on CI runners; using language: python with additional_dependencies lets pre-commit install ty directly.
Self was added to typing in 3.11. Since annotations are lazy (PEP 563), Self is only needed by the type checker, not at runtime.
The built-in fixtures for monkeypatch, recwarn, and tmp_path now delegate to near-verbatim copies of pytest's implementations under `python/karva/_vendor/`, with pytest's MIT copyright notice appended to the repository LICENSE following the ruff vendoring pattern. This fixes several latent issues that came from the hand-rolled versions: tmp_path no longer leaks a directory per test because TempPathFactory reuses a numbered basetemp with retention, setattr/delattr now raise TypeError on misuse (matching pytest's documented contract), and setenv warns via the standard PytestWarning-equivalent path when handed a non-string value. The integration tests for monkeypatch that previously asserted AttributeError on bad argument shapes are updated to expect TypeError, and one test covering a validation pytest doesn't perform is removed rather than kept as a karva-only divergence. The recwarn snapshots are regenerated because WarningsRecorder reports its class name via catch_warnings' default repr instead of the old `<WarningsChecker object>`.
Adds a new integration test file that wraps near-verbatim copies of the test bodies from pytest's testing/test_monkeypatch.py as karva TestContext cases. Each test is grouped so that related pytest assertions run in one subprocess invocation, and each group uses `command_no_parallel()` for deterministic snapshot output. Tests that depended on pytest's `pytester`, the `_pytest.config` internals, or the legacy `pkg_resources` namespace-package machinery are intentionally skipped because those features are either absent in karva or were dropped when vendoring MonkeyPatch. The file header lists the substitutions applied (pytest.raises -> karva.raises, @pytest.mark. parametrize -> @karva.tags.parametrize, MonkeyPatch() -> karva.MockEnv()) so future diffs against upstream stay auditable.
Framework fixtures (monkeypatch, tmp_path, capsys, ...) were previously discovered into a bare Vec<DiscoveredFixture>, stored on PackageRunner, threaded into every RuntimeFixtureResolver, and checked via a linear fallback scan inside get_dependent_fixtures. That parallel lookup path drifted away from the HasFixtures/find_fixture walk the rest of the runner uses, cost three touch-points for one conceptual feature, and left framework fixtures invisible to everything except explicit test parameters. Framework fixtures are now packaged into a synthetic DiscoveredModule and attached to the session root's new `framework_module` slot during discovery. The HasFixtures impl on DiscoveredPackage checks the user conftest first and falls through to the framework module, so the existing find_fixture walk reaches them naturally at the end of the parent chain. A user conftest.py fixture with the same name still wins because the configuration module is consulted first at every level. This lets the runner delete the framework_fixtures field, drop the extra parameter on RuntimeFixtureResolver, and remove the fallback branch in get_dependent_fixtures. Net ~40 fewer lines of Rust, and one trait delegation point replaces a scattered special case.
The files in `python/karva/_vendor/` are near-verbatim copies of pytest source. Running ruff's `SIM`/`B` checks against them surfaces 11 upstream style idioms (`False if ... else True`, `try`/`except`/`pass`, nested `if`s, `warnings.warn` without stacklevel) that we cannot rewrite without diverging from the vendored commit. Add a lint exclude so ruff format still runs on these files but lint rules do not. The existing `ruff format` pass stays green, ty continues to check them, and upstream upgrades can be replayed without hitting lint churn.
`_CapLog.set_level` previously only captured the level of the first logger it touched, so `caplog.set_level(DEBUG, "pkg.a")` followed by `caplog.set_level(INFO, "pkg.b")` would leak `pkg.b`'s new level past teardown. It now records the original level of every logger touched (keyed by logger name, including `None` for the root logger) and restores all of them when the caplog fixture tears down. Also records the original handler level so repeat `set_level` calls cannot drift it. `HasFixtures::auto_use_fixtures` for `DiscoveredPackage` was extending framework and user conftest autouse fixtures into the same list, so a future autouse fixture defined in both would run twice. It now dedupes by unqualified function name with user conftest winning — matching the `get_fixture` precedence contract. Drop the dead `@fixture def recwarn` and `@fixture def monkeypatch` stubs from the vendor modules. The framework-fixture discoverer only parses `python/karva/_builtins.py`, so these stubs were never reachable; their real fixture wrappers already live in `_builtins.py` and import `MockEnv` / `WarningsRecorder` from the vendor modules. Document the remaining substantive deviations (MockEnv's added `__enter__`/`__exit__`, the absence of the `monkeypatch` fixture wrapper) in the vendor module headers.
The rewrite of built-in fixtures from Rust to Python (and the framework fixture unification) was thin on new test coverage. This adds two new integration test files that exercise the rewritten surface end-to-end through the karva test runner: `more_builtins.rs` (29 tests) adds precedence regressions (user `conftest.py` tmp_path and monkeypatch fixtures shadowing the framework versions), a multi-logger `caplog.set_level` regression that would have caught the teardown bug fixed in the previous commit, extra capsys / recwarn / tmp_path_factory edge cases, `MockEnv()` used directly as a context manager, monkeypatch undo coverage for classmethod and staticmethod descriptors and for setattr on a missing attribute, and a missing-fixture diagnostic snapshot so diagnostic regressions become visible. `pytest_vendor_tests.rs` (12 tests) is a near-verbatim port of the pytest test classes that exercise the vendored modules directly: `TestWarningsRecorderChecker` for `WarningsRecorder` lifecycle, plus `TestNumberedDir`, `TestRmRf`, and `TestTmpPathHandler::test_mktemp` adapted to run through karva. These tests give us coverage of the pathlib helpers (`make_numbered_dir`, `cleanup_numbered_dir`, `ensure_deletable`, `maybe_delete_a_numbered_dir`, `rm_rf`) which were previously exercised only transitively via `tmp_path`/`tmp_path_factory`. Net: 794 tests -> 834 tests.
MatthewMckee4
force-pushed
the
worktree-glowing-baking-flurry
branch
from
03c5865 to
c2205a4
Compare
…llowing `discover_framework_fixtures` previously short-circuited via `.ok()?` on every step of its error chain, so an unimportable `karva._builtins`, a missing `__file__`, a non-UTF-8 path, an unreadable source file, a parser failure, or a per-function `try_from_function` failure would all silently leave the session with an empty framework module. Every branch now logs at `tracing::warn!` with the specific failure so an operator seeing "fixture not found" errors can trace back to the root cause. `NormalizedFixture::call` and `PackageRunner::execute_test_variant` both built their pyo3 kwargs dict with `let _ = py_dict.set_item(...)`, which hid non-hashable / non-convertible fixture arguments behind a confusing "missing required keyword-only argument" from the downstream call. The `?` now propagates the `PyErr` from the fixture result path (into the normal `FixtureCallError` chain) and from the test result path (into the normal `classify_test_result` branch).
`_pytest_monkeypatch.py::delattr` now uses `name is _NOTSET` to match pytest's identity check instead of the wider `isinstance(name, _NotSetType)` widening we had before. The narrower check rejects any future second enum variant that happens to be a `_NotSetType` value. `_pytest_pathlib.py` drops the unused `_ignore_error`, `_IGNORED_ERRORS`, and `_IGNORED_WINERRORS` helpers. These were copied from CPython's pathlib for pytest's symlink-scan code path, which karva does not vendor. The vendor header documents the removal. New `python/karva/_vendor/README.md` pins the upstream pytest commit (`8ecf49ec2`), lists every vendored module with its upstream path, and documents the re-sync procedure. Updates here must keep `ruff.lint. exclude` in `pyproject.toml` in sync so upstream diffs stay auditable. Two new regression tests in `more_builtins.rs`: - `test_session_autouse_can_depend_on_framework_fixture` exercises a user-defined session-scope autouse fixture that depends on the framework `tmp_path_factory`. Before the autouse gate fix in the companion commit this setup could not resolve the framework dependency at session scope. - `test_derive_importpath_regression_from_pytest_issue_1338` is karva's equivalent of pytest's `test_issue1338_name_resolving` (an external dependency on `requests` substituted with `os.path.defpath`, which exercises the same `importlib.import_module` branch inside `derive_importpath`).
The upstream-pin table in `python/karva/_vendor/README.md` failed CI's `mdformat` step because the column padding on the vendored commits does not round-trip through mdformat's table formatter — CI rewrote it with minimally-padded columns on every run. Rewriting as a bullet list avoids the issue entirely and matches the existing project convention from the memory note on CONTRIBUTING.md (tables trigger MD060). Same content, no functional change. The ordered-list formatting for the re-sync steps is already mdformat-compatible (1./1./1. style) from the earlier commit.
MatthewMckee4
added
needs-decision
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Built-in fixtures (
capsys,capfd,capsysbinary,capfdbinary,caplog,tmp_path,tmpdir,tmp_path_factory,tmpdir_factory,monkeypatch,recwarn)were previously implemented as Rust
#[pyclass]structs spread across six filesin
crates/karva_test_semantic/src/extensions/fixtures/builtins/(~1,800 lines).They are now plain Python generator functions in
python/karva/_builtins.py,discovered at runtime the same way as user-defined fixtures.
The key mechanism is
discover_framework_fixtures: it importskarva._builtinsviapy.import("karva._builtins"), reads__file__to locate the source, parses it withruff, and creates
DiscoveredFixtureobjects for each@fixture-decorated function.These are passed to
RuntimeFixtureResolveras a fallback after user-defined fixtures.Because all fixtures now follow the same code path,
NormalizedFixturecollapsesfrom a two-variant enum (
BuiltIn/UserDefined) to a plain struct, removing thebranching throughout
package_runner.rsandfixture_resolver.rs.MockEnvmovesfrom a Rust
#[pyclass]to a pure-Python class and is now imported fromkarva._builtinsinstead ofkarva._karva.Test Plan
All existing integration tests pass unchanged — fixture behaviour is preserved by
the Python re-implementation. The
test_mock_env,test_capsys,test_tmp_path,test_caplog, andtest_recwarnintegration tests were the primary verification.