feat: Add Gradle task to check auto-import documentation sync#7782
Draft
margaretkennedy wants to merge 1 commit intodeephaven:mainfrom
Draft
feat: Add Gradle task to check auto-import documentation sync#7782margaretkennedy wants to merge 1 commit intodeephaven:mainfrom
margaretkennedy wants to merge 1 commit intodeephaven:mainfrom
Conversation
Adds checkAutoImportDocs task to engine-table that: - Uses Java reflection to extract methods from QueryLibraryImportsDefaults.statics() - Parses markdown docs to find documented methods - Compares and reports any mismatches - Fails build if methods are missing from docs Usage: ./gradlew :engine-table:checkAutoImportDocs This is lightweight and can be added to pre-merge checks.
margaretkennedy
added
documentation
Contributor
No docs changes detected for bd4e9f7 |
margaretkennedy
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a Gradle verification task to :engine-table to check that the auto-imported query language documentation remains synchronized with the Java auto-import definitions in QueryLibraryImportsDefaults.
Changes:
- Register new
checkAutoImportDocsGradle task in:engine-table. - Use reflection to enumerate public static methods/fields from
QueryLibraryImportsDefaults.statics(). - Parse the markdown docs under
docs/python/reference/query-language/query-library/auto-imported/and compare against expected symbols, failing the build on missing docs.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| inputs.file sourceFile | ||
| inputs.dir docsDir | ||
| outputs.upToDateWhen { true } |
There was a problem hiding this comment.
outputs.upToDateWhen { true } marks the task as always up-to-date, so checkAutoImportDocs will typically be skipped and never actually verify anything. For a verification task, remove the custom up-to-date predicate or set it to false so it always runs (or declare a real output and let Gradle determine up-to-date status).
Suggested change
| outputs.upToDateWhen { true } |
Comment on lines
+155
to
+221
| def classLoader = new URLClassLoader( | ||
| sourceSets.main.runtimeClasspath.collect { it.toURI().toURL() } as URL[], | ||
| getClass().classLoader | ||
| ) | ||
|
|
||
| // Load QueryLibraryImportsDefaults and get the statics | ||
| def importsClass = classLoader.loadClass('io.deephaven.engine.table.lang.impl.QueryLibraryImportsDefaults') | ||
| def instance = importsClass.getDeclaredConstructor().newInstance() | ||
| def staticsSet = instance.statics() as Set<Class<?>> | ||
|
|
||
| // Collect all public static methods from statics classes | ||
| def expectedMethods = new TreeSet<String>() | ||
| staticsSet.each { clazz -> | ||
| clazz.declaredMethods.each { method -> | ||
| if (java.lang.reflect.Modifier.isPublic(method.modifiers) && | ||
| java.lang.reflect.Modifier.isStatic(method.modifiers)) { | ||
| expectedMethods.add(method.name) | ||
| } | ||
| } | ||
| // Also get public static fields (constants) | ||
| clazz.declaredFields.each { field -> | ||
| if (java.lang.reflect.Modifier.isPublic(field.modifiers) && | ||
| java.lang.reflect.Modifier.isStatic(field.modifiers)) { | ||
| expectedMethods.add(field.name) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // Parse markdown docs to find documented methods | ||
| def documentedMethods = new TreeSet<String>() | ||
| docsDir.eachFileMatch(~/.*.md/) { file -> | ||
| if (file.name == 'index.md') return | ||
| file.eachLine { line -> | ||
| // Parse table rows: | TYPE | name | signature | description | | ||
| def matcher = line =~ /^\|\s*(FUNCTION|CONSTANT|CLASS)\s*\|\s*(\w+)\s*\|/ | ||
| if (matcher.find()) { | ||
| documentedMethods.add(matcher.group(2)) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // Compare | ||
| def missingInDocs = expectedMethods - documentedMethods | ||
| def extraInDocs = documentedMethods - expectedMethods | ||
|
|
||
| // Filter out known exceptions (methods that shouldn't be documented) | ||
| def excludedMethods = ['getClass', 'hashCode', 'equals', 'toString', 'notify', 'notifyAll', 'wait'] as Set | ||
| missingInDocs.removeAll(excludedMethods) | ||
|
|
||
| def hasErrors = false | ||
|
|
||
| if (!missingInDocs.isEmpty()) { | ||
| logger.error("Methods in QueryLibraryImportsDefaults.statics() but NOT documented:") | ||
| missingInDocs.each { logger.error(" - $it") } | ||
| hasErrors = true | ||
| } | ||
|
|
||
| if (!extraInDocs.isEmpty()) { | ||
| logger.warn("Methods documented but NOT in QueryLibraryImportsDefaults.statics():") | ||
| extraInDocs.each { logger.warn(" - $it") } | ||
| } | ||
|
|
||
| if (hasErrors) { | ||
| throw new GradleException("Auto-import documentation is out of sync. Run the generator script to update docs.") | ||
| } | ||
|
|
||
| logger.lifecycle("Auto-import documentation check passed. ${documentedMethods.size()} methods documented.") |
There was a problem hiding this comment.
The URLClassLoader created here is never closed. Since URLClassLoader is Closeable, leaving it open can leak file handles / keep jars locked (notably on Windows) in long-lived Gradle daemons. Wrap usage in a try/finally (or Groovy withCloseable) and close it after loading/reflecting.
Suggested change
| def classLoader = new URLClassLoader( | |
| sourceSets.main.runtimeClasspath.collect { it.toURI().toURL() } as URL[], | |
| getClass().classLoader | |
| ) | |
| // Load QueryLibraryImportsDefaults and get the statics | |
| def importsClass = classLoader.loadClass('io.deephaven.engine.table.lang.impl.QueryLibraryImportsDefaults') | |
| def instance = importsClass.getDeclaredConstructor().newInstance() | |
| def staticsSet = instance.statics() as Set<Class<?>> | |
| // Collect all public static methods from statics classes | |
| def expectedMethods = new TreeSet<String>() | |
| staticsSet.each { clazz -> | |
| clazz.declaredMethods.each { method -> | |
| if (java.lang.reflect.Modifier.isPublic(method.modifiers) && | |
| java.lang.reflect.Modifier.isStatic(method.modifiers)) { | |
| expectedMethods.add(method.name) | |
| } | |
| } | |
| // Also get public static fields (constants) | |
| clazz.declaredFields.each { field -> | |
| if (java.lang.reflect.Modifier.isPublic(field.modifiers) && | |
| java.lang.reflect.Modifier.isStatic(field.modifiers)) { | |
| expectedMethods.add(field.name) | |
| } | |
| } | |
| } | |
| // Parse markdown docs to find documented methods | |
| def documentedMethods = new TreeSet<String>() | |
| docsDir.eachFileMatch(~/.*.md/) { file -> | |
| if (file.name == 'index.md') return | |
| file.eachLine { line -> | |
| // Parse table rows: | TYPE | name | signature | description | | |
| def matcher = line =~ /^\|\s*(FUNCTION|CONSTANT|CLASS)\s*\|\s*(\w+)\s*\|/ | |
| if (matcher.find()) { | |
| documentedMethods.add(matcher.group(2)) | |
| } | |
| } | |
| } | |
| // Compare | |
| def missingInDocs = expectedMethods - documentedMethods | |
| def extraInDocs = documentedMethods - expectedMethods | |
| // Filter out known exceptions (methods that shouldn't be documented) | |
| def excludedMethods = ['getClass', 'hashCode', 'equals', 'toString', 'notify', 'notifyAll', 'wait'] as Set | |
| missingInDocs.removeAll(excludedMethods) | |
| def hasErrors = false | |
| if (!missingInDocs.isEmpty()) { | |
| logger.error("Methods in QueryLibraryImportsDefaults.statics() but NOT documented:") | |
| missingInDocs.each { logger.error(" - $it") } | |
| hasErrors = true | |
| } | |
| if (!extraInDocs.isEmpty()) { | |
| logger.warn("Methods documented but NOT in QueryLibraryImportsDefaults.statics():") | |
| extraInDocs.each { logger.warn(" - $it") } | |
| } | |
| if (hasErrors) { | |
| throw new GradleException("Auto-import documentation is out of sync. Run the generator script to update docs.") | |
| } | |
| logger.lifecycle("Auto-import documentation check passed. ${documentedMethods.size()} methods documented.") | |
| new URLClassLoader( | |
| sourceSets.main.runtimeClasspath.collect { it.toURI().toURL() } as URL[], | |
| getClass().classLoader | |
| ).withCloseable { classLoader -> | |
| // Load QueryLibraryImportsDefaults and get the statics | |
| def importsClass = classLoader.loadClass('io.deephaven.engine.table.lang.impl.QueryLibraryImportsDefaults') | |
| def instance = importsClass.getDeclaredConstructor().newInstance() | |
| def staticsSet = instance.statics() as Set<Class<?>> | |
| // Collect all public static methods from statics classes | |
| def expectedMethods = new TreeSet<String>() | |
| staticsSet.each { clazz -> | |
| clazz.declaredMethods.each { method -> | |
| if (java.lang.reflect.Modifier.isPublic(method.modifiers) && | |
| java.lang.reflect.Modifier.isStatic(method.modifiers)) { | |
| expectedMethods.add(method.name) | |
| } | |
| } | |
| // Also get public static fields (constants) | |
| clazz.declaredFields.each { field -> | |
| if (java.lang.reflect.Modifier.isPublic(field.modifiers) && | |
| java.lang.reflect.Modifier.isStatic(field.modifiers)) { | |
| expectedMethods.add(field.name) | |
| } | |
| } | |
| } | |
| // Parse markdown docs to find documented methods | |
| def documentedMethods = new TreeSet<String>() | |
| docsDir.eachFileMatch(~/.*.md/) { file -> | |
| if (file.name == 'index.md') return | |
| file.eachLine { line -> | |
| // Parse table rows: | TYPE | name | signature | description | | |
| def matcher = line =~ /^\|\s*(FUNCTION|CONSTANT|CLASS)\s*\|\s*(\w+)\s*\|/ | |
| if (matcher.find()) { | |
| documentedMethods.add(matcher.group(2)) | |
| } | |
| } | |
| } | |
| // Compare | |
| def missingInDocs = expectedMethods - documentedMethods | |
| def extraInDocs = documentedMethods - expectedMethods | |
| // Filter out known exceptions (methods that shouldn't be documented) | |
| def excludedMethods = ['getClass', 'hashCode', 'equals', 'toString', 'notify', 'notifyAll', 'wait'] as Set | |
| missingInDocs.removeAll(excludedMethods) | |
| def hasErrors = false | |
| if (!missingInDocs.isEmpty()) { | |
| logger.error("Methods in QueryLibraryImportsDefaults.statics() but NOT documented:") | |
| missingInDocs.each { logger.error(" - $it") } | |
| hasErrors = true | |
| } | |
| if (!extraInDocs.isEmpty()) { | |
| logger.warn("Methods documented but NOT in QueryLibraryImportsDefaults.statics():") | |
| extraInDocs.each { logger.warn(" - $it") } | |
| } | |
| if (hasErrors) { | |
| throw new GradleException("Auto-import documentation is out of sync. Run the generator script to update docs.") | |
| } | |
| logger.lifecycle("Auto-import documentation check passed. ${documentedMethods.size()} methods documented.") | |
| } |
|
|
||
| // Parse markdown docs to find documented methods | ||
| def documentedMethods = new TreeSet<String>() | ||
| docsDir.eachFileMatch(~/.*.md/) { file -> |
There was a problem hiding this comment.
The file-match regex ~/.*.md/ treats . as “any character”, so it can match unintended filenames (and is not equivalent to “ends with .md”). Use an escaped dot / end anchor (e.g., ~/.*\.md$/) to reliably select markdown files.
Suggested change
| docsDir.eachFileMatch(~/.*.md/) { file -> | |
| docsDir.eachFileMatch(~/.*\.md$/) { file -> |
Comment on lines
+189
to
+215
| def matcher = line =~ /^\|\s*(FUNCTION|CONSTANT|CLASS)\s*\|\s*(\w+)\s*\|/ | ||
| if (matcher.find()) { | ||
| documentedMethods.add(matcher.group(2)) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // Compare | ||
| def missingInDocs = expectedMethods - documentedMethods | ||
| def extraInDocs = documentedMethods - expectedMethods | ||
|
|
||
| // Filter out known exceptions (methods that shouldn't be documented) | ||
| def excludedMethods = ['getClass', 'hashCode', 'equals', 'toString', 'notify', 'notifyAll', 'wait'] as Set | ||
| missingInDocs.removeAll(excludedMethods) | ||
|
|
||
| def hasErrors = false | ||
|
|
||
| if (!missingInDocs.isEmpty()) { | ||
| logger.error("Methods in QueryLibraryImportsDefaults.statics() but NOT documented:") | ||
| missingInDocs.each { logger.error(" - $it") } | ||
| hasErrors = true | ||
| } | ||
|
|
||
| if (!extraInDocs.isEmpty()) { | ||
| logger.warn("Methods documented but NOT in QueryLibraryImportsDefaults.statics():") | ||
| extraInDocs.each { logger.warn(" - $it") } | ||
| } |
There was a problem hiding this comment.
documentedMethods currently includes CLASS rows from java.md / data-types.md, but expectedMethods only contains static method/field names from statics(). This makes extraInDocs extremely noisy (hundreds/thousands of class names) and the lifecycle message “X methods documented” is misleading. Consider either (a) parsing only FUNCTION|CONSTANT rows for this task, or (b) also validating QueryLibraryImportsDefaults.classes() separately and reporting each category independently.
Comment on lines
+166
to
+172
| def expectedMethods = new TreeSet<String>() | ||
| staticsSet.each { clazz -> | ||
| clazz.declaredMethods.each { method -> | ||
| if (java.lang.reflect.Modifier.isPublic(method.modifiers) && | ||
| java.lang.reflect.Modifier.isStatic(method.modifiers)) { | ||
| expectedMethods.add(method.name) | ||
| } |
There was a problem hiding this comment.
expectedMethods tracks only method.name, so it can’t detect missing/extra overloads (the docs list many overloads per name, e.g., multiple array / concat rows). If the goal is to ensure the documentation is fully in sync, consider comparing by a stable signature key (method name + parameter arity/types) rather than name alone.
devinrsmith
reviewed
Mar 20, 2026
| } | ||
|
|
||
| if (hasErrors) { | ||
| throw new GradleException("Auto-import documentation is out of sync. Run the generator script to update docs.") |
Member
There was a problem hiding this comment.
I'm confused a bit; there's a script to update the documentation. Why don't we just run the auto-import script into a build directory, and then compare the contents of what was built to what is in source control?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds checkAutoImportDocs task to engine-table that:
Usage: ./gradlew :engine-table:checkAutoImportDocs
This is lightweight and can be added to pre-merge checks.
Currently: You must run the task manually
Future options:
Add check.dependsOn(checkAutoImportDocs) → runs with every ./gradlew check
Add to CI workflow → runs on every PR