Documenting all existing rules

[dayantur]

Currently, there is no central, user-accessible documentation for all validator rules outside of the codebase itself. The new rule registry implemented in #1246 (and potential AI template workflow for rule creation in #1248) makes it even more important to document rules systematically. Attempting to collect rules manually (e.g., PR #851) isn't scalable, so an automated approach is needed.

Proposed Documentation Strategy:

• Fundamental Constraints and Ranges (Class 1 rules - C1; can only be validations):
  Document these via the range_values / range_description fields in the JSON schema.
  These constraints should automatically appear in the documentation for each parameter.
• All Other Rules (Class 2 rules - C2; can be either adjustments or validations or warnings);
  Collect and document other rules in a dedicated .rst doc file. Use docstrings from each rule function, ensuring they follow a consistent, Numpy-style format and correctly describe the rule. Organise the documentation using rule IDs and categories/scripts in Add a draft rules registry to the validator with two test rules #1246.

Proposed Implementation Steps:

(The first two steps can be carried out in parallel.)

A. Extract all parameter ranges and field constraints from existing Pydantic fields and include them in parameter descriptions.
B. Review and update all rule docstrings to follow the Numpy docstring standard and ensure accuracy.
C. Create a rulebook.rst file and automatically collect/aggregate all rules into this documentation file.

@MatthewPaskin @sunt05 let me know your thoughts!

[sunt05]

Thanks for putting this together — the direction is right. Auto-generating documentation from code is exactly how we should handle this, and you've correctly identified that the manual CSV approach in #851 doesn't scale.

A few technical points to sharpen the proposal:

Step A — which source of truth for ranges?

The codebase currently has two separate systems for range constraints: the legacy checker_rules_indiv.json (used by _check.py for DataFrame-level validation) and the Pydantic Field validators in the new data model. When you mention documenting ranges "via the range_values / range_description fields in the JSON schema," it's worth clarifying which system you mean.

We're planning to deprecate the legacy JSON checker — the new Pydantic-based validation pipeline is the way forward. That said, checker_rules_indiv.json does contain useful range rules that haven't been migrated yet. So before we retire it, we should audit what's in there and port any rules worth keeping into the new system. Documenting the JSON file itself isn't the right investment; ranges should live in the Pydantic models, where generate_datamodel_rst.py can already pick them up.

Step B — agreed, this is clearly needed

Most rule docstrings are currently one-liners (e.g. "Validate required physics parameters."). The validate_irrigation_doy function is the only one with a proper NumPy-style docstring. Standardising the rest is a well-defined task and a good starting point.

Step C — unstated prerequisite

The current RulesRegistry.add_rule() decorator only stores {rule_id: function} — no metadata (category, severity, target parameters). To auto-generate a useful rulebook.rst, the registry would need extending, something like:

@RulesRegistry.add_rule("land_cover", category="LAND_COVER", severity="ERROR")

This is a code change that should be scoped as an explicit step before C, since it affects the registry design from #1246.

C1/C2 classification

The split is a useful conceptual framework, but it would help to ground it in the actual code architecture rather than the CSV taxonomy from #851. In practice: C1 maps to Pydantic model constraints (Field validators, ge=, le=, etc.), and C2 maps to RulesRegistry rules. Making this mapping explicit would clarify what "documenting C1 rules" actually means in implementation terms.

[dayantur]

Hi @sunt05 - thanks for taking the time to read and comment this!

Regarding Step A: I would say Pydantic fields - via generate_datamodel_rst.py (still have to look at that, but good that we have already this pipeline implemented)
Regarding Step C: I agree - this might be very helpful in the future, and useful to migrate also the remaining rules across codebase (adjustments in phase B are still not in the registry, as well as the SUEWSConfig validators)
RegardinC1/C2 classification: I agree here as well - it might be even easier to stick with Fields/Rules naming. But we can re-check this and decide the best naming convention when we get there.

[sunt05]

Thanks for confirming, @dayantur — sounds like we're aligned.

I'd suggest we tackle this in the following order:

1. Extend RulesRegistry.add_rule() to accept metadata (category, severity, target parameters) — this is the prerequisite I flagged, and it unblocks the auto-generated docs in step 3.
2. Standardise rule docstrings to NumPy-style across all 12 rules, using validate_irrigation_doy as the template. This can run in parallel with step 1.
3. Auto-generate rulebook.rst from the enriched registry — once 1 and 2 are done.

For Step A (Field constraints in docs): generate_datamodel_rst.py already extracts Pydantic ge=/le= constraints into the RST output, so this is largely covered. We should do a quick audit against checker_rules_indiv.json to catch any gaps before we retire that file, but it's not a blocker.

Happy for you to start on step 2 (docstrings) straight away — that's self-contained and well-defined.

[dayantur]

Hi @sunt05 - I will cover 2, my feeling on timing is that can be done by end of this week :)

Is the Pydantic field extractor able to pull also from gt, lt? I suppose yes?

[sunt05]

Yes — generate_datamodel_rst.py already handles all four: gt (>), ge (>=), lt (<), le (<=). They're extracted from Pydantic field metadata and rendered as a :Constraints: line in the RST output. So you're good to go.