Skip to content

chore(nimbus): add GHA workflows for desktop integration tests#14965

Open
jaredlockhart wants to merge 11 commits intomainfrom
14583
Open

chore(nimbus): add GHA workflows for desktop integration tests#14965
jaredlockhart wants to merge 11 commits intomainfrom
14583

Conversation

@jaredlockhart
Copy link
Collaborator

@jaredlockhart jaredlockhart commented Mar 18, 2026 •
edited
Loading

Because

  • The desktop integration tests currently only run on CircleCI
  • We are migrating all CI jobs from CircleCI to GitHub Actions (EXP-6320)

This commit

  • Adds GHA workflows for all 5 desktop integration test suites:
    • integration-nimbus-ui.yml — Nimbus UI tests (-m nimbus_ui)
    • integration-remote-settings-launch.yml — Remote Settings launch test
    • integration-remote-settings-all.yml — Remote Settings all workflows
    • integration-desktop-enrollment.yml — Desktop enrollment tests
    • integration-desktop-targeting.yml — Desktop targeting (Release + Beta + Nightly matrix)
  • All workflows use the existing setup-cached-build composite action for Docker layer caching via Google Artifact Registry
  • All run on ubuntu-24.04 with the same make targets and Docker Compose setup as CircleCI
  • All upload HTML test reports as GitHub Actions artifacts
  • Targeting tests use a matrix strategy for Release/Beta/Nightly channels

Fixes #14583
Fixes #14584
Fixes #14585
Fixes #14586
Fixes #14587

@jaredlockhart jaredlockhart changed the title chore(nimbus): add GHA workflow for Desktop Nimbus UI integration tests chore(nimbus): add GHA workflows for desktop integration tests Mar 18, 2026
@jaredlockhart jaredlockhart force-pushed the 14583 branch 2 times, most recently from 579ae9c to 1ec63e4 Compare March 19, 2026 17:44
jaredlockhart and others added 9 commits March 19, 2026 15:51
@jaredlockhart @claude
chore(nimbus): add GHA workflows for desktop integration tests
cc8583f 
Because

* The desktop integration tests currently only run on CircleCI
* We are migrating all CI jobs from CircleCI to GitHub Actions (EXP-6320)

This commit

* Adds GHA workflows for all 5 desktop integration test suites:
  - `integration-nimbus-ui.yml` — Nimbus UI tests (`-m nimbus_ui`)
  - `integration-remote-settings-launch.yml` — Remote Settings launch test
  - `integration-remote-settings-all.yml` — Remote Settings all workflows
  - `integration-desktop-enrollment.yml` — Desktop enrollment tests
  - `integration-desktop-targeting.yml` — Desktop targeting (Release + Beta + Nightly matrix)
* All workflows use the existing `setup-cached-build` composite action for
  Docker layer caching via Google Artifact Registry
* All run on `ubuntu-24.04` with the same make targets and Docker Compose
  setup as CircleCI
* All upload HTML test reports as GitHub Actions artifacts
* Targeting tests use a matrix strategy for Release/Beta/Nightly channels

Fixes #14583
Fixes #14584
Fixes #14585
Fixes #14586
Fixes #14587

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jaredlockhart @claude
fix(nimbus): split remote-settings-all GHA workflow into 3 matrix jobs
3fd32e2 
Because

* The previous `-k FIREFOX_DESKTOP` filter selected cirrus_enrollment
  tests that require the Cirrus container, which isn't started by
  up_prod_detached without CIRRUS=1
* CircleCI splits this job into 3 parallel nodes by marker:
  remote_settings_experiments, remote_settings_rollouts,
  remote_settings_live_updates

This commit

* Converts the single job into a matrix strategy with 3 marker-based
  splits matching CircleCI's parallelism
* Gives each matrix job a unique artifact name

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jaredlockhart @claude
chore(nimbus): use xl runners for GHA integration tests
85ee655 
Because

* GHA integration tests on ubuntu-24.04 (4 vCPU) run 40-80% slower
  than equivalent CircleCI jobs on large/xlarge instances
* Target is ~10min per job to match CircleCI performance

This commit

* Bumps all 5 integration test workflows from ubuntu-24.04 (4 vCPU)
  to ubuntu-24.04-xl (8 vCPU, 32GB RAM)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jaredlockhart @claude
fix(nimbus): revert to ubuntu-24.04 runners for integration tests
db6a97c 
Because

* ubuntu-24.04-xl runners are not available in the mozilla org,
  causing jobs to queue indefinitely waiting for a runner

This commit

* Reverts all integration test workflows back to ubuntu-24.04

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jaredlockhart @claude
chore(nimbus): add run-integration-test composite action with retry
293cb79 
Because

* Transient infra failures (DNS resolution, upstream 500s) cause
  integration test jobs to fail before pytest even starts
* The retry logic was duplicated across all 5 integration workflows

This commit

* Adds .github/actions/run-integration-test composite action that
  wraps the make command with configurable retry attempts and
  handles test report artifact upload
* Migrates all 5 integration test workflows to use the new action

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jaredlockhart @claude
chore(nimbus): add test striping for parallel GHA integration jobs
c6d8043 
Because

* Desktop targeting tests (222 cases) take 19-27min per channel
* Desktop enrollment tests (6 cases) take 19min
* GHA runners are 4 vCPU so we can't increase xdist parallelism

This commit

* Adds --split/--splits pytest options via conftest.py hook that
  deterministically stripes tests by modulo across GHA matrix nodes
* Splits targeting into 2 nodes per channel (6 total, was 3)
* Splits enrollment into 2 nodes (was 1)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jaredlockhart @claude
style(nimbus): add missing blank line in conftest.py
c7b41a0 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jaredlockhart @claude
fix(nimbus): sort test items by nodeid before splitting
fab5aaf 
Because

* pytest doesn't guarantee stable collection order across runs
* Without deterministic ordering, the modulo split could assign
  different tests to different nodes on each run

This commit

* Sorts items by nodeid before applying the modulo split

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jaredlockhart @claude
chore(nimbus): add composite action path to integration workflow trig…
931d339 
…gers

Because

* Changes to .github/actions/run-integration-test/ should trigger
  integration test workflows to verify the action still works

This commit

* Adds .github/actions/run-integration-test/** to path filters in
  all 5 integration test workflows

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
jaredlockhart and others added 2 commits March 19, 2026 16:00
@jaredlockhart @claude
chore(nimbus): set BUILDKIT_PROGRESS=plain for CI-friendly Docker output
a08f608 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jaredlockhart @claude
chore(nimbus): set COMPOSE_ANSI=never to suppress pull progress bars …
8f3b624 
…in CI

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@relud relud relud approved these changes

@freshstrangemusic freshstrangemusic Awaiting requested review from freshstrangemusic freshstrangemusic is a code owner

@mikewilli mikewilli Awaiting requested review from mikewilli mikewilli is a code owner

@yashikakhurana yashikakhurana Awaiting requested review from yashikakhurana yashikakhurana is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

2 participants

@jaredlockhart @relud