Build cluster & configure cluster updates#51
Open
cnealen wants to merge 10 commits intodevelopmentfrom
Open
Conversation
Major changes: - Clone nwm-automation-scripts repo (was missing) - Explicitly pull all 8 required Docker images from ghcr.io/ngwpc - Create individual Singularity containers for all 8 images with timestamped builds - Update static files setup to include module parameter files from nwm-msw-mgr, BMI forcing templates from ngen-forcing, and verification data from nwm-verf - Fix ngencerf-server configuration to use correct path (cerfServer/.env-override) - Add logrotate cron job setup for ngencerf-server logs - Replace build_cluster.sh call with explicit container build steps
Added -y/--non-interactive flag to skip file editing prompts. AWS credentials are prompted at the start of script execution and stored in variables.
Automatically appends MSWM_TAG, NGEN_FORCING_TAG, and DATA_ASSIMILATION_TAG to .env-override with default value 'development'. Works in both interactive and non-interactive modes.
Displays a 5-second warning message at the end of script execution to remind users that TAG variables are set to 'development' and should only be changed for release candidates or official releases.
In non-interactive mode, users can now paste the entire AWS credentials block (all three export statements) at once, then press Ctrl+D to submit. The script will parse the credentials from the export statements automatically.
- Displays warning at start of script before AWS credentials - Changed from 5-second sleep to Y/n confirmation prompt - User can cancel configuration if they don't want to proceed - Removed duplicate warning from end of script
…f the cluster, while build_cluster.sh handles all Docker/Singluarity image operations.
2. Updated static files path 3. cleaned up static file steps
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--forcing-types=bmi,sfincs), improved error handling and dependency tag promptingChanges
build_cluster.sh
--branch=REPO:REFfor per-repo branch/commit hash specification--branch-default=REFfor global default ref--forcing-types=bmi,sfincsfor selecting which forcing images to buildis_commit_hash()to detect commit hashes and skipgit pullfor them--ngen-forcing-tagand--ngen-tagCLI flags for dependency tag specificationset_image_source_defaults()overwriting explicit--sourceflagsngen-forcingarithmetic issue)set -einteraction with[[ ! -v ]]conditionalsdockerfile_for_forcing_type()andimage_name_for_forcing_type()helpersconfigure_cluster.sh
Other
full_cleanup_cluster.shfor full cluster resetshowLatestReleaseTags.shto show latest tag across reposTest plan
bash -n build_cluster.sh— syntax check passes./build_cluster.sh --build-type=development ngen— development build works./build_cluster.sh --build-type=feature --branch=ngen:a1b2c3d ngen— commit hash support./build_cluster.sh --build-type=development --forcing-types=bmi,sfincs ngen-forcing— forcing type selection./build_cluster.sh --build-type=development --source=ngen:pull ngen— per-repo source selection./configure_cluster.sh— cluster setup workflow