Missingdep and dynamic dependencies

[moritzx22]

Extend 'ninja -t missingdeps' to Support Dyndeps

Currently, the ninja -t missingdeps subtool does not take dynamic dependencies (dyndep) into account. This PR extends the functionality of missingdeps to properly follow and evaluate dyndep relationships.

The example below demonstrates such a case:
A missing dependency is reported even though it is correctly satisfied through a dynamically generated input generated.h.

Show full `build.ninja` file

rule catdep
  depfile = a.d
  dyndep = dyndep.dd
  command = cat $in > $out && printf 'a.o: generated.h\n' > a.d
build a.o: catdep a.c || dyndep.dd

rule touch
  command = touch $out
build generated.h X: touch in

rule dd
  command = touch dyndep.dd && printf 'ninja_dyndep_version = 1\nbuild a.o: dyndep | X\n' > $out
build dyndep.dd: dd

As of today, running ninja -t missingdeps on the first example produces:

$ ninja -t missingdeps
Missing dep: a.o uses generated.h (generated by touch)
Processed 4 nodes.
Error: There are 1 missing dependency paths.
1 targets had depfile dependencies on 1 distinct generated inputs (from 1 rules)  without a non-depfile dep path to the generator.
There might be build flakiness if any of the targets listed above are built alone, or not late enough, in a clean output directory.

This is a false positive, since generated.h is actually provided via a dyndep relationship. However, because dyndeps are currently ignored by the missingdeps tool, the dependency path is not detected.

Changes in This PR

Extend the missingdeps algorithm to:

• Traverse and evaluate dynamic dependencies (dyndep)
• Incorporate dynamically discovered inputs into dependency analysis
• Eliminate false positives caused by dependencies that are only known after dyndep evaluation
  Running the same example with the proposed

$ ninja -t missingdeps
Processed 4 nodes.
No missing dependencies on generated files found.

The false positive is resolved, as the tool now correctly follows dyndep relationships and recognizes that generated.h is properly generated before it is consumed.

Missingdyndeps

This PR also introduces the concept of "missingdyndeps", analogous to missingdep from depfiles.

A missingdyndep occurs when:

• A build edge depends on dynamically generated outputs,
• But the dyndep-producing edge is not guaranteed to run before the dependent edge.

In a well-defined build:

• All dynamic outputs must be generated before they are consumed,
• Even if the corresponding dyndep file has not yet been loaded.

Example missingdyndep:

The following example demonstrates a missing dyndep situation:

Show full `build.ninja` file

rule dd
  command = touch dyndep.dd && echo 'ninja_dyndep_version = 1\nbuild out | generated.h: dyndep\n' > dyndep.dd

rule dyn_touch
  command = touch generated.h out
  dyndep = dyndep.dd

rule request
  command = cat $in > $out

build dyndep.dd: dd
build out: dyn_touch || dyndep.dd
build b.out: request generated.h

In this setup, it is not guaranteed that the dyn_touch edge runs before the request edge if the dyndep file has not yet been loaded. As a result, generated.h may be consumed before it is produced.

The detection of missingdyndeps is integrated into the existing missingdeps tool. When such a case is encountered, it is reported alongside the usual output.

For the example above, the output becomes:

$ ninja -t missingdeps
Processed 4 nodes.
No missing dep found
Missing Dyndep: 'generated.h' is an input to 'b.out' and a dyndep-output of 'dyndep.dd'
There might be build flakiness if any of the targets listed above are built alone, or not late enough, in a clean output directory.

This indicates that while no traditional missingdeps were found, a missing dyndep condition was still detected.

Resolve example

This can be resolved by explicitly enforcing the correct build order, for example:

...
#build b.out: request generated.h

#Option 1:
build b.out: request generated.h || out

#Option 2:
#build b.out: request generated.h || dyndep.dd

Both approaches ensure that the dynamic dependency is properly respected and that generated.h is generated before it is used.

Here are the graphs without missingdyndeps

graph click to show

Option 1:

Option 2:

[jhasse]

Ignore the codespell stuff ... I'm not sure if that shouldn't just be removed from CI.