Decoupling HDF5 to EMOD3D format conversion from from generate_velocity_model by Copilot · Pull Request #101 · ucgmsim/workflow

Copilot · 2026-03-31T01:08:47Z

This pull request introduces a new workflow step to convert velocity model HDF5 files to EMOD3D binary format, decoupling this conversion from the velocity model generation.

It also updates the workflow configuration and command-line interface to support this new step.

Additionally, the velocity model generation script is refactored to remove the embedded conversion logic, streamlining responsibilities and improving maintainability.

Workflow enhancements:

Added a new convert_vm_hdf5_to_emod3d workflow step in cylc/flow.cylc to convert HDF5 velocity models to EMOD3D format as a separate, parallelized step (using 4 threads)
Registered the convert-vm-hdf5-to-emod3d command in the CLI by updating pyproject.toml.

New script and CLI command:

Introduced the workflow/scripts/convert_vm_hdf5_to_emod3d.py script, providing a standalone CLI for converting HDF5 velocity models to EMOD3D binary files, with logging and documentation.
Relaxed the existence check for the velocity_model_output argument in generate_velocity_model to allow for pre-existing output directories.

…ommands Agent-Logs-Url: https://github.com/ucgmsim/workflow/sessions/9633754d-83a7-4c4e-aafd-0714bd366c75 Co-authored-by: sungeunbae <466989+sungeunbae@users.noreply.github.com>

Agent-Logs-Url: https://github.com/ucgmsim/workflow/sessions/d3d34c6d-6b1e-4488-9140-71842281e8ad Co-authored-by: sungeunbae <466989+sungeunbae@users.noreply.github.com>

…_velocity_model_hdf5() Agent-Logs-Url: https://github.com/ucgmsim/workflow/sessions/0734c2e6-6b40-4570-8e34-328c03f1fbb2 Co-authored-by: sungeunbae <466989+sungeunbae@users.noreply.github.com>

…df5 CLI commands Agent-Logs-Url: https://github.com/ucgmsim/workflow/sessions/6426afbd-1729-4121-ba7e-9b318ded0171 Co-authored-by: sungeunbae <466989+sungeunbae@users.noreply.github.com>

…command reference Agent-Logs-Url: https://github.com/ucgmsim/workflow/sessions/c68f4200-ca6d-432c-8d71-1b53fee120db Co-authored-by: sungeunbae <466989+sungeunbae@users.noreply.github.com>

Copilot

Pull request overview

This PR introduces a dedicated workflow/CLI step to convert NZCVM-produced velocity model HDF5 output into EMOD3D binary format, removing the embedded conversion logic from generate_velocity_model and updating workflow configuration accordingly.

Changes:

Refactors generate-velocity-model to stop doing HDF5→EMOD3D conversion inline (intended to leave conversion to a new dedicated step).
Adds a new convert-vm-hdf5-to-emod3d Typer CLI script and registers it as a project script entrypoint.
Updates the Cylc flow to run the new conversion step as a separate task.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
`workflow/scripts/generate_velocity_model.py`	Removes inline HDF5→EMOD3D conversion/move behavior from the NZCVM path.
`workflow/scripts/convert_vm_hdf5_to_emod3d.py`	Adds a standalone CLI wrapper around `velocity_modelling.tools.convert_hdf5_to_emod3d`.
`pyproject.toml`	Registers the new CLI command entrypoint.
`cylc/flow.cylc`	Adds a new conversion task and changes the velocity model generation invocation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

workflow/scripts/generate_velocity_model.py

workflow/scripts/convert_vm_hdf5_to_emod3d.py

cylc/flow.cylc

…tence check; fix nproc in cylc Agent-Logs-Url: https://github.com/ucgmsim/workflow/sessions/46e17c6d-85c9-4369-ba43-c3a94218ffd3 Co-authored-by: sungeunbae <466989+sungeunbae@users.noreply.github.com>

…:ucgmsim/workflow into copilot/refactor-run-nzcvm-into-commands

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…:ucgmsim/workflow into copilot/refactor-run-nzcvm-into-commands

workflow/scripts/convert_vm_hdf5_to_emod3d.py

Initial plan

6f48b34

Copilot AI assigned Copilot and sungeunbae Mar 31, 2026

Copilot started work on behalf of sungeunbae March 31, 2026 01:08 View session

feat: split run_nzcvm into generate/convert steps with separate CLI c…

0686b29

…ommands Agent-Logs-Url: https://github.com/ucgmsim/workflow/sessions/9633754d-83a7-4c4e-aafd-0714bd366c75 Co-authored-by: sungeunbae <466989+sungeunbae@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Refactor run_nzcvm into separate commands for HDF5 generation~~ Split velocity model generation into separate checkpointed Cylc steps Mar 31, 2026

Copilot finished work on behalf of sungeunbae March 31, 2026 01:18

Copilot AI requested a review from sungeunbae March 31, 2026 01:18

Copilot started work on behalf of sungeunbae March 31, 2026 02:00 View session

refactor: extract _prepare_nzvm_config helper to remove duplication

8b0a17e

Agent-Logs-Url: https://github.com/ucgmsim/workflow/sessions/d3d34c6d-6b1e-4488-9140-71842281e8ad Co-authored-by: sungeunbae <466989+sungeunbae@users.noreply.github.com>

Copilot AI changed the title ~~Split velocity model generation into separate checkpointed Cylc steps~~ Deduplicate realisation setup between generate_velocity_model and generate_velocity_model_hdf5 Mar 31, 2026

Copilot finished work on behalf of sungeunbae March 31, 2026 02:05

Copilot started work on behalf of sungeunbae March 31, 2026 02:26 View session

Copilot AI and others added 3 commits March 31, 2026 02:32

refactor: add emod3d_convert switch to run_nzcvm(); simplify generate…

a6d707f

…_velocity_model_hdf5() Agent-Logs-Url: https://github.com/ucgmsim/workflow/sessions/0734c2e6-6b40-4570-8e34-328c03f1fbb2 Co-authored-by: sungeunbae <466989+sungeunbae@users.noreply.github.com>

Minimal change for separating HDF5 to EMOD3D conversion

44d60fe

Minimal change for separating HDF5 to EMOD3D conversion

7a88e12

Copilot AI changed the title ~~Deduplicate realisation setup between generate_velocity_model and generate_velocity_model_hdf5~~ Add emod3d_convert switch to run_nzcvm() to eliminate duplication with generate_velocity_model_hdf5() Mar 31, 2026

Copilot finished work on behalf of sungeunbae March 31, 2026 02:36

Minimal change needed

6d60046

Copilot started work on behalf of sungeunbae March 31, 2026 02:41 View session

feat: add generate_velocity_model_hdf5() and convert-velocity-model-h…

49397c6

…df5 CLI commands Agent-Logs-Url: https://github.com/ucgmsim/workflow/sessions/6426afbd-1729-4121-ba7e-9b318ded0171 Co-authored-by: sungeunbae <466989+sungeunbae@users.noreply.github.com>

Copilot AI changed the title ~~Add emod3d_convert switch to run_nzcvm() to eliminate duplication with generate_velocity_model_hdf5()~~ Add generate_velocity_model_hdf5() and convert-velocity-model-hdf5 CLI to expose HDF5→EMOD3D split to Singularity Mar 31, 2026

Copilot finished work on behalf of sungeunbae March 31, 2026 02:50

sungeunbae added 3 commits March 31, 2026 16:01

Added convert-vm-hdf5-to-emod3d script

9a5eeff

Merge conflict resolved

d7496f2

Removed copilot's attempt to add unnecessary apps

483d2d7

Copilot started work on behalf of sungeunbae March 31, 2026 03:27 View session

fix: add module docstring to convert_vm_hdf5_to_emod3d.py, fix stale …

fa703db

…command reference Agent-Logs-Url: https://github.com/ucgmsim/workflow/sessions/c68f4200-ca6d-432c-8d71-1b53fee120db Co-authored-by: sungeunbae <466989+sungeunbae@users.noreply.github.com>

Copilot AI changed the title ~~Add generate_velocity_model_hdf5() and convert-velocity-model-hdf5 CLI to expose HDF5→EMOD3D split to Singularity~~ Expose HDF5→EMOD3D conversion as a standalone CLI command for Cylc split workflow Mar 31, 2026

Copilot finished work on behalf of sungeunbae March 31, 2026 03:34

Copilot AI changed the title ~~Split velocity model generation into HDF5 and EMOD3D conversion Cylc tasks~~ Remove emod3d_convert switch from generate_velocity_model — HDF5-only output Mar 31, 2026

Copilot finished work on behalf of sungeunbae March 31, 2026 04:02

Copilot started work on behalf of sungeunbae March 31, 2026 20:20 View session

Copilot finished work on behalf of sungeunbae March 31, 2026 20:24

sungeunbae added 2 commits April 1, 2026 09:25

Removed convert-emod3d switch

94b03a3

Local changes before merge

7e921aa

sungeunbae changed the title ~~Remove emod3d_convert switch from generate_velocity_model — HDF5-only output~~ Decoupling HDF5 to EMOD3D format conversion from from generate_velocity_model Mar 31, 2026

sungeunbae requested review from AndrewRidden-Harper and lispandfound March 31, 2026 23:08

sungeunbae marked this pull request as ready for review March 31, 2026 23:08

Copilot AI review requested due to automatic review settings March 31, 2026 23:08

Copilot started reviewing on behalf of sungeunbae March 31, 2026 23:08 View session

Copilot AI reviewed Mar 31, 2026

View reviewed changes

workflow/scripts/generate_velocity_model.py Show resolved Hide resolved

workflow/scripts/convert_vm_hdf5_to_emod3d.py Outdated Show resolved Hide resolved

cylc/flow.cylc Outdated Show resolved Hide resolved

cylc/flow.cylc Outdated Show resolved Hide resolved

Copilot started work on behalf of sungeunbae April 1, 2026 00:51 View session

fix: make velocity_model_output optional in NZCVM mode; add HDF5 exis…

c8fcf7c

…tence check; fix nproc in cylc Agent-Logs-Url: https://github.com/ucgmsim/workflow/sessions/46e17c6d-85c9-4369-ba43-c3a94218ffd3 Co-authored-by: sungeunbae <466989+sungeunbae@users.noreply.github.com>

Copilot finished work on behalf of sungeunbae April 1, 2026 00:56

sungeunbae and others added 4 commits April 1, 2026 14:27

Merge branch 'copilot/refactor-run-nzcvm-into-commands' of github.com…

876ac37

…:ucgmsim/workflow into copilot/refactor-run-nzcvm-into-commands

Apply suggestions from code review

4ec37e5

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Tidy up flow.cylc

d63392b

Merge branch 'copilot/refactor-run-nzcvm-into-commands' of github.com…

31324d8

…:ucgmsim/workflow into copilot/refactor-run-nzcvm-into-commands

lispandfound approved these changes Apr 1, 2026

View reviewed changes

workflow/scripts/convert_vm_hdf5_to_emod3d.py Outdated Show resolved Hide resolved

lispandfound requested changes Apr 1, 2026

View reviewed changes

workflow/scripts/convert_vm_hdf5_to_emod3d.py Outdated Show resolved Hide resolved

sungeunbae added 2 commits April 1, 2026 14:50

Simplified convert_vm_hdf5_to_emod3d() signature

0da1641

Removed unused import

ed2731e

sungeunbae requested a review from lispandfound April 1, 2026 01:53

lispandfound approved these changes Apr 1, 2026

View reviewed changes

AndrewRidden-Harper approved these changes Apr 1, 2026

View reviewed changes

sungeunbae merged commit 554d3b1 into pegasus Apr 1, 2026
7 checks passed

sungeunbae deleted the copilot/refactor-run-nzcvm-into-commands branch April 1, 2026 02:03

sungeunbae restored the copilot/refactor-run-nzcvm-into-commands branch April 1, 2026 02:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Decoupling HDF5 to EMOD3D format conversion from from generate_velocity_model#101

Decoupling HDF5 to EMOD3D format conversion from from generate_velocity_model#101
sungeunbae merged 25 commits intopegasusfrom
copilot/refactor-run-nzcvm-into-commands

Copilot AI commented Mar 31, 2026 •

edited by sungeunbae

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

Copilot AI commented Mar 31, 2026 • edited by sungeunbae Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Copilot AI commented Mar 31, 2026 •

edited by sungeunbae

Loading