Skip to content

ci: use custom VM images#6303

Draft
v1v wants to merge 11 commits intomainfrom
feature/use-custom-vm-images
Draft

ci: use custom VM images#6303
v1v wants to merge 11 commits intomainfrom
feature/use-custom-vm-images

Conversation

@v1v
Copy link
Member

@v1v v1v commented Feb 6, 2026

What is the problem this PR solves?

Use custom VM images

How does this PR solve the problem?

Ensure we know what tools we use and pin the version - then we will be able to automatically bump the versions using the PR review process.

How to test this PR locally

Design Checklist

  • I have ensured my design is stateless and will work when multiple fleet-server instances are behind a load balancer.
  • I have or intend to scale test my changes, ensuring it will work reliably with 100K+ agents connected.
  • I have included fail safe mechanisms to limit the load on fleet-server: rate limiting, circuit breakers, caching, load shedding, etc.

Checklist

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool

Related issues

@v1v
Copy link
Member Author

v1v commented Feb 6, 2026

@mergify
Copy link
Contributor

mergify bot commented Feb 6, 2026

This pull request does not have a backport label. Could you fix it @v1v? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-./d./d is the label to automatically backport to the 8./d branch. /d is the digit
  • backport-active-all is the label that automatically backports to all active branches.
  • backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
  • backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

FIPS: "true"
agents:
provider: "gcp"
image: "${IMAGE_UBUNTU_X86_64}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@v1v Apparently the FIPS are using the same images, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's only for the packaging, as far as I see at 8bb8c74

The tests run using the FIPS images.

Copy link
Contributor

@fr4nc1sc0-r4m0n fr4nc1sc0-r4m0n Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what's the reason for that? Wouldn't it make more sense to build and test over the same images?

Even with that, the ones that are failing are the packaging pipelines https://buildkite.com/elastic/fleet-server-package-mbp/builds/3243

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know, #4574 introduced it.

@fr4nc1sc0-r4m0n
Copy link
Contributor

Seems that are flaky test because in each execution a different ones fails. Retrying

env:
IMAGE_UBUNTU_X86_64: "family/core-ubuntu-2204"
IMAGE_UBUNTU_ARM_64: "core-ubuntu-2204-aarch64"
IMAGE_UBUNTU_X86_64: "platform-ingest-fleets-ubuntu-2204-1770381074"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running agent pre-command hook, I'm not sure what's the reason for using that particular pre-command hook, but IIRC, fleet-server does not need it.

Can you confirm if the VM Images used are not copying the pre-command hook?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been trying to find any reference from the fleet-server images but I found nothing, so I'd say that no copy of the hooks/base/pre-command is happening in this images

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1770381074 is from February (2026-02-06 12:31:14 UTC), but there are new commits in the ci-agent-images. Can you generate the new VM images and update the timestamp here?

Copy link
Contributor

@fr4nc1sc0-r4m0n fr4nc1sc0-r4m0n Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure:

Let's see if it works.

Copy link
Contributor

@fr4nc1sc0-r4m0n fr4nc1sc0-r4m0n Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this execution the E2E test have passed:

@mergify
Copy link
Contributor

mergify bot commented Mar 3, 2026

This pull request is now in conflicts. Could you fix it @v1v? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b feature/use-custom-vm-images upstream/feature/use-custom-vm-images
git merge upstream/main
git push upstream feature/use-custom-vm-images

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants