WA-CI-012: Improve CI messaging for Bundler frozen-mode lockfile mismatch

[kitcommerce]

Summary

• Detect common Bundler frozen/deployment-mode lockfile mismatch failures in CI and emit a high-signal GitHub Actions error annotation with the remediation.

Client impact

None expected.

Verification Plan

1. In a PR, introduce a Gemfile/Gemfile.lock mismatch (e.g. edit Gemfile without updating Gemfile.lock).
2. Run CI (or re-run the failed job).
3. Observe a single ::error:: annotation explaining the frozen-mode lockfile mismatch and instructing to run bundle install and commit Gemfile.lock.

[kitcommerce]

Architecture Review

Verdict: PASS

The script is appropriately placed in script/ci/ — a good subdirectory convention for CI-specific tooling. Responsibility is well-scoped: detect lockfile mismatch, emit a GitHub Actions ::error:: annotation, exit 1.

The YAML change is architecturally sound: splitting continue-on-error logic into separate experimental vs non-experimental branches makes the intent explicit and prevents swallowing non-experimental failures silently. The indirection (hint script always exits 1) is a clean way to fail the job with enriched messaging.

No coupling concerns. Follows the established script/ pattern.

[kitcommerce]

Simplicity Review

Verdict: PASS

The 48-line script does one thing: re-run bundle install, capture output, grep for known lockfile error patterns, and emit an annotation. The log-to-temp-file approach is standard and avoids stdout/stderr entanglement. set -euo pipefail is correct defensive practice.

The pattern of always exiting 1 ensures the hint step never accidentally passes, which keeps the logic simple and explicit. No unnecessary complexity observed.

[kitcommerce]

Security Review

Verdict: PASS

No shell injection vectors identified. The script re-runs bundle install in a controlled CI environment — no user-supplied input is interpolated into commands. The ::error:: annotation content is sourced from bundle output; while a malicious gem could theoretically craft error messages to inject into annotations, this is an extremely low-risk scenario in the context of a controlled CI lockfile check.

RUNNER_TEMP fallback to /tmp is safe. set -euo pipefail prevents silent failures. All variable expansions appear properly handled.

[kitcommerce]

Rails Conventions Review

Verdict: PASS (N/A)

This PR contains only shell scripts and YAML — no Ruby code. Rails conventions do not apply. Confirmed.

[kitcommerce]

Rails Security Review

Verdict: PASS

Findings

No Rails security findings. This PR modifies only CI workflow configuration (.github/workflows/ci.yml) and adds a diagnostic bash script (script/ci/bundler_lockfile_hint). No Rails application code (controllers, models, views, routes, or Ruby source) is touched — zero Rails security attack surface affected.

Recommendations

None.

[kitcommerce]

Database Review

Verdict: PASS

Findings

No database changes in this PR. The diff consists entirely of CI workflow YAML () and a new shell script (). There are no migrations, schema changes, model query patterns, or ActiveRecord code to evaluate.

Recommendations

None.

[kitcommerce]

Test Quality Review

Verdict: PASS_WITH_NOTES

Summary

This is a CI tooling-only change (bash script + YAML). No Ruby unit tests are added or modified, which is expected — the test quality question here is whether the script's logic is adequately verifiable and whether the verification plan is sound.

Findings

MEDIUM — Regex pattern coverage is untested and brittle
The core detection logic in script/ci/bundler_lockfile_hint relies on a grep -Eq pattern list to identify frozen/deployment-mode lockfile mismatch messages. These patterns are the heart of the feature. If any bundler version changes its error message wording slightly, or if the patterns miss a real-world variant, the hint silently falls through to the generic message — providing no improvement over the baseline. There are no automated tests to verify the patterns match actual bundler output.

LOW — Verification plan is entirely manual
The verification plan (introduce a mismatch, run CI, observe annotation) is operationally reasonable but not repeatable or automated. A bash testing framework like bats-core could mock bundle to emit known error strings and assert on both exit codes and stdout, making the regex patterns and branch logic continuously testable without a full CI run.

What's Done Well

• set -euo pipefail is present — good defensive practice for CI scripts
• Each branch of the script has clear inline comments explaining intent
• The script re-runs bundle install to capture output, rather than trying to parse an upstream log — this is robust and avoids log-file coordination issues
• The fallback message (non-lockfile-mismatch case) also exits non-zero and emits an annotation — no silent failure
• CI YAML correctly gates the hint step on steps.bundle.outcome != 'success' && !matrix.experimental so experimental failures won't generate false-positive lockfile annotations

Recommendations

1. Consider adding a test/ci/bundler_lockfile_hint.bats (or equivalent) with at least three cases: (a) bundle exit 0 re-run, (b) bundle fails with a known lockfile mismatch message, (c) bundle fails with an unrecognized message. This makes the regex patterns auditable and protects against bundler version drift.
2. Document the source of the pattern strings (e.g., which bundler version / error class they come from) so future maintainers know what to update when patterns drift.

Conclusion

The script is well-structured and non-destructive. The absence of automated tests is a real gap for the regex-based detection logic, but given the narrow scope of this CI tooling change and the low blast radius (worst case: annotation doesn't fire, CI still fails), this does not rise to CHANGES_REQUIRED. Shipping with a follow-up bats test tracked as a low-priority item is acceptable.

[kitcommerce]

Rails Security Review

Verdict: PASS

Findings

No Rails security findings. This PR modifies only CI workflow configuration (.github/workflows/ci.yml) and adds a diagnostic bash script (script/ci/bundler_lockfile_hint). No Rails application code (controllers, models, views, routes, or Ruby source) is touched — zero Rails security attack surface affected.

Recommendations

None.

[kitcommerce]

Accessibility Review

Verdict: PASS (N/A)

CI configuration and shell scripts — no user-facing UI or accessibility surface.

---

🦾 Automated accessibility review — Kit

[kitcommerce]

Frontend Review

Verdict: PASS (N/A)

CI YAML and shell scripts — no frontend surface.

---

🖥️ Automated frontend review — Kit