Skip to content

feat(commands): add sync schema config tool#7689

Merged
acwhite211 merged 16 commits intomainfrom
issue-7564
Mar 18, 2026
Merged

feat(commands): add sync schema config tool#7689
acwhite211 merged 16 commits intomainfrom
issue-7564

Conversation

@grantfitzsimmons
Copy link
Member

@grantfitzsimmons grantfitzsimmons commented Feb 8, 2026

Fixes #7564 and starts to fix #7426

Adds a tool to create new schema config records for tables and fields present in the data model that do not have schema records.

This still needs to be invoked in a migration (or ideally when we run the 'self-heal' function)...

  • Added new command sync_schema_config_fields.py to identify (and create if) missing schema config tables and fields for a given discipline. Can be run in verbose mode to see every addition, and can identify without immediately adding.
  • Introduced utility functions find_missing_schema_config_fields and create_missing_schema_config_fields in update_schema_config.py to support the management command and allow programmatic access to these operations.

It should look something like this (plus many {table}Id fields):

Discipline: Vertebrate Zoology (id=3)

Missing containers:
- DNASequencingRunAttachment
- LocalityUpdate
- LocalityUpdateRowResult

Missing fields:
- Attachment: spDataSetAttachments
- CollectionObject: collectionObjectType
- DNASequencingRunAttachment: ordinal, remarks, timestampCreated, timestampModified, version, attachment, createdByAgent, dnaSequencingRun, modifiedByAgent
- Discipline: tectonicUnitTreeDef
- Locality: collectingEvents
- LocalityUpdate: status, timestampcreated, timestampmodified, recordset, collection, specifyuser, createdbyagent, modifiedbyagent
- LocalityUpdateRowResult: rownumber, result, localityupdate
- SpQueryField: isStrict

Checklist

  • Self-review the PR after opening it to make sure the changes look good and
    self-explanatory (or properly documented)
  • Add relevant issue to release milestone
  • Add pr to documentation list
  • Add automated tests
  • Add a reverse migration if a migration is present in the PR

Testing instructions

  • Identify the discipline ID for any discipline in a database
  • Inside the container, run the following, replacing 13 with the ID of the discipline you wish to test this on
    • See all missing tables and fields:
     ve/bin/python manage.py sync_schema_config_fields --discipline-id 13
    
    • Create all missing table and field records for the schema config:
     ve/bin/python manage.py sync_schema_config_fields --discipline-id 13 --apply
    
    • Create all missing table and field records for the schema config with logging for each addition:
     ve/bin/python manage.py sync_schema_config_fields --discipline-id 13 --apply --verbose
    

Fixes #7564 and starts to fix #7426

Adds a tool to create new schema config records for tables and fields present in the data model that do not have schema records.
Triggered by 38a8223 on branch refs/heads/issue-7564
@grantfitzsimmons
Copy link
Member Author

@acwhite211 Your thoughts on this would be appreciated!

@grantfitzsimmons grantfitzsimmons added this to the 7.13.0 milestone Feb 11, 2026
@CarolineDenis CarolineDenis modified the milestones: 7.13.0, 7.12.1 Feb 19, 2026
Copy link
Member

@acwhite211 acwhite211 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job! Looks good. I added a commit to make it a little more efficient with bulk fetching. No problems during testing, the Django command seems to be able to accomplish it's goal.

@grantfitzsimmons grantfitzsimmons marked this pull request as ready for review February 23, 2026 17:40
@grantfitzsimmons grantfitzsimmons requested a review from a team February 23, 2026 17:49
Base automatically changed from issue-2931-2 to main February 26, 2026 13:27
Copy link
Contributor

@alesan99 alesan99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Identify the discipline ID for any discipline in a database
  • Inside the container, run the following, replacing 13 with the ID of the discipline you wish to test this on
    • See all missing tables and fields:
    • Create all missing table and field records for the schema config:
    • Create all missing table and field records for the schema config with logging for each addition:

Identifies missing schema and confirms its been added once its run with --apply 👍👍
Image

@alesan99 alesan99 requested a review from a team March 17, 2026 18:20
Removed console log for missing ranks check.
@CarolineDenis CarolineDenis linked an issue Mar 18, 2026 that may be closed by this pull request
5 tasks
@acwhite211 acwhite211 merged commit 2f3e277 into main Mar 18, 2026
14 checks passed
@acwhite211 acwhite211 deleted the issue-7564 branch March 18, 2026 16:23
@github-project-automation github-project-automation bot moved this from 📋Back Log to ✅Done in General Tester Board Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅Done

Development

Successfully merging this pull request may close these issues.

Some tables do not have Schema Config records, making them unhideable Add "ID" Field to Schema Config

4 participants