Fix journald input for journalctl versions < 242 by omitting --boot all

[belimawr]

Proposed commit message

journalctl versions < 242 do not support the `--boot all` flag that
newer versions require to read all boot messages, this commit fixes
this problem by omitting the flag because versins < 242 returns
messages from all boots without the need to set `--boot all`

The journal files used for tests are updated making them compatible
with journalctl >= 239.

The integration test TestJournaldInputReadsMessagesFromAllBoots is
added to ensure the journald input can always read messages form all
boots. This test is intended to be run manually on VMs where the
journal has less than 50 000 messages.

Some fixes to the life-cycle of `journalctl` are implemented because they
were required for the tests to be stable.

GenAI-Assisted: Yes
Human-Reviewed: Yes
Tool: Cursor-CLI, Model: GPT-5.3 Codex Extra High Fast
Tool: Cursor-CLI, Model:  Claude 4.6 Opus (Thinking)

Note for reviewers

There are still some small issues with the process management in filebeat/input/journald/pkg/journalctl/journalctl.go, I'll fix them in a follow up PR (based on this PR and linked here). This PR has grown too large already to include other fixes.

The only fixes unrelated to the original issue I included here are the ones that caused failures on CI or during my tests.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

## Disruptive User Impact

Author's Checklist

• Create follow up PR to fix the "process management" issues. The branch with the fixes: https://github.com/belimawr/beats/tree/fix-journalctl-life-cycle. I'll create a PR once this one is merged.

How to test this PR locally

1) Run existing tests (unit + integration)

These cover the new journalctl argument behavior by version.

cd filebeat
go test ./input/journald/... -v
mage BuildSystemTestBinary
go test -count=1 -v -tags=integration ./tests/integration -run=TestJournald

2) Use the provided VMs to test different versions of journalctl

export VAGRANT_CWD="<Absolute path to Beats repo>/filebeat/input/journald"
export VAGRANT_VAGRANTFILE="$VAGRANT_CWD/Vagrantfile"

• VAGRANT_VAGRANTFILE makes Vagrant always use filebeat/input/journald/Vagrantfile.
• VAGRANT_CWD defines where Vagrant will save its state and which folder to use for relative paths

Bring up VM matrix and verify versions

vagrant up 239 250 # or any other version/VM you want to test
vagrant status

Manually validate the new all-boots integration test

TestJournaldInputReadsMessagesFromAllBoots is intentionally skipped by
default, you'll need to export JOURNALD_MANUAL_TEST=1 for the test
to run.

Inside one of the VMs (recommended: 239 and 250):

# SSH into the VM
vagrant ssh 239
journalctl --list-boots --no-pager # ensure ther are multiple boots in the journal

# Copy Beats repo, Filebeat won't run correctly on a mounted volume
cp -r /vagrant/ ./beats

cd beats/filebeat
mage BuildSystemTestBinary
JOURNALD_MANUAL_TEST=1 go test -tags integration ./tests/integration -run TestJournaldInputReadsMessagesFromAllBoots -v

Expected:

• Test sees more than one boot.
• Test computes target event count from the two oldest boots.
• Test passes with at least 2 distinct journald.host.boot_id values.

If there is only one boot, create another boot (sudo reboot), reconnect, and
run again.

3) Manual test

Run Filebeat with the following configuration:

Details

filebeat.inputs:
  - type: journald
    id: test--boot-all

output.discard:
  enabled: true

logging:
  to_stderr: true

Look for the log entries with message: Journalctl command. Paths relative to chroot (if set). They will contain a field process.command_line with the whole journalctl command.

You can use the following command to filter the logs and format them with jq:

go run . --path.home=$PWD 2>&1 | grep -i "Journalctl command." --line-buffered|jq   

You should see logs like:

{
  "log.level": "info",
  "@timestamp": "2026-03-12T13:14:25.465-0400",
  "log.logger": "input.journald.reader",
  "log.origin": {
    "function": "github.com/elastic/beats/v7/filebeat/input/journald.(*journald).Run.NewFactory.func1",
    "file.name": "journalctl/journalctl.go",
    "file.line": 86
  },
  "message": "Journalctl command. Paths relative to chroot (if set)",
  "service.name": "filebeat",
  "id": "test--boot-all",
  "input_source": "LOCAL_SYSTEM_JOURNAL",
  "path": "LOCAL_SYSTEM_JOURNAL",
  "input_id": "test--boot-all",
  "process.command_line": "journalctl --version",
  "process.chroot": "",
  "ecs.version": "1.6.0"
}
{
  "log.level": "info",
  "@timestamp": "2026-03-12T13:14:25.467-0400",
  "log.logger": "input.journald.reader.journalctl-runner",
  "log.origin": {
    "function": "github.com/elastic/beats/v7/filebeat/input/journald.(*journald).Run.NewFactory.func1",
    "file.name": "journalctl/journalctl.go",
    "file.line": 86
  },
  "message": "Journalctl command. Paths relative to chroot (if set)",
  "service.name": "filebeat",
  "id": "test--boot-all",
  "input_source": "LOCAL_SYSTEM_JOURNAL",
  "path": "LOCAL_SYSTEM_JOURNAL",
  "input_id": "test--boot-all",
  "process.command_line": "journalctl --utc --output=json --no-pager --all --follow --after-cursor s=e82795fad4ce42b79fb3da0866d91f7e;i=15502a0;b=e224754c921445279ded5949d3a0dfc9;m=1820a8d806;t=64a18fd40f98c;x=a138e845a98ffe14 --boot all",                    
  "process.chroot": "",
  "ecs.version": "1.6.0"
}

Related issues

• Closes Journald input is not compatible with Journald < v242 #48152
• Replaces Fix journald input for journalctl versions < 242 by omitting --boot all. #49413

## Use cases
## Screenshots
## Logs

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[philippkahr]

LGTM! can't. comment on the code really. Functionality looks like exactly what we need :)

[AndersonQ]

@belimawr I think you missed to link the follow-up PR

[AndersonQ]

You could make the vagrant machines to install go and mage during their creation. They're meant to test filebeat anyway

[belimawr]

@belimawr I think you missed to link the follow-up PR

I linked the branch with the changes. Because both branches are in my fork, I cannot open a PR on this repo and a merge commit was making the diff to list more changes than only my commit. So to keep things simple I just linked the branch. I'll use it to create a PR as soon as this one is merged.

[coderabbitai]

Actionable comments posted: 1

♻️ Duplicate comments (1)

filebeat/input/journald/input.go (1)

381-382: ⚠️ Potential issue | 🔴 Critical

The nolint is masking a real timestamp overflow.

Line 382 still narrows data.RealtimeTimestamp from uint64 to int64 before time.UnixMicro. If that value exceeds math.MaxInt64, the timestamp wraps to a bogus value; suppressing gosec does not make the conversion safe.

#!/bin/bash
rg -n -C2 'RealtimeTimestamp|UnixMicro|nolint:gosec' filebeat/input/journald/input.go

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@filebeat/input/journald/input.go` around lines 381 - 382, The code narrows
data.RealtimeTimestamp (uint64) to int64 for time.UnixMicro which can overflow;
remove the nolint and instead check and handle the range before converting:
compute secs := data.RealtimeTimestamp / 1_000_000 and use nsec :=
(data.RealtimeTimestamp % 1_000_000) * 1_000, ensure secs fits in int64 (compare
against uint64(math.MaxInt64)); if it overflows clamp or handle as appropriate,
then call time.Unix(int64(secs), int64(nsec)) to build the timestamp without
unsafe uint64->int64 narrowing (update the code around the time.UnixMicro usage
that references data.RealtimeTimestamp).

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@filebeat/tests/integration/journald_test.go`:
- Around line 218-223: The test currently derives expectedMessages from
oldestBoot and secondOldestBoot but only asserts that the first N events contain
two distinct boot IDs, which can pass without including those specific boots;
change the assertions to explicitly check that bootIDs (the set/list extracted
from the consumed events) contains oldestBoot.BootID and secondOldestBoot.BootID
(in addition to existing distinctness checks). Locate where oldestBoot and
secondOldestBoot are used to build expectedMessages and where bootIDs is
asserted (around the countBootEntries call and the subsequent event assertions)
and add two explicit contains assertions for oldestBoot.BootID and
secondOldestBoot.BootID; apply the same explicit-contains fixes to the analogous
assertions in the block spanning lines 237-263.

---

Duplicate comments:
In `@filebeat/input/journald/input.go`:
- Around line 381-382: The code narrows data.RealtimeTimestamp (uint64) to int64
for time.UnixMicro which can overflow; remove the nolint and instead check and
handle the range before converting: compute secs := data.RealtimeTimestamp /
1_000_000 and use nsec := (data.RealtimeTimestamp % 1_000_000) * 1_000, ensure
secs fits in int64 (compare against uint64(math.MaxInt64)); if it overflows
clamp or handle as appropriate, then call time.Unix(int64(secs), int64(nsec)) to
build the timestamp without unsafe uint64->int64 narrowing (update the code
around the time.UnixMicro usage that references data.RealtimeTimestamp).

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c2a87096-c5b6-4726-9f22-7660ab2733d7

📥 Commits

Reviewing files that changed from the base of the PR and between e8db4fe and faeff8d.

📒 Files selected for processing (6)

• changelog/fragments/1772726727-fix-journald-args.yaml
• filebeat/input/journald/README.md
• filebeat/input/journald/Vagrantfile
• filebeat/input/journald/input.go
• filebeat/input/journald/pkg/journalctl/reader_test.go
• filebeat/tests/integration/journald_test.go

🚧 Files skipped from review as they are similar to previous changes (3)

• filebeat/input/journald/Vagrantfile
• changelog/fragments/1772726727-fix-journald-args.yaml
• filebeat/input/journald/pkg/journalctl/reader_test.go

[github-actions]

@Mergifyio backport 8.19 9.2 9.3

[mergify]

backport 8.19 9.2 9.3

✅ Backports have been created

Details

• #49525 [8.19](backport #49445) Fix journald input for journalctl versions < 242 by omitting --boot all has been created for branch 8.19 but encountered conflicts

Cherry-pick of f6662a9 has failed:

On branch mergify/bp/8.19/pr-49445
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit f6662a962.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   changelog/fragments/1772726727-fix-journald-args.yaml
	modified:   filebeat/input/journald/README.md
	new file:   filebeat/input/journald/Vagrantfile
	modified:   filebeat/input/journald/input.go
	modified:   filebeat/input/journald/input_filtering_test.go
	new file:   filebeat/input/journald/journald.conf
	modified:   filebeat/input/journald/pkg/journalctl/reader.go
	modified:   filebeat/input/journald/testdata/binary.journal.gz
	new file:   filebeat/input/journald/testdata/convert_export_to_journal.sh
	modified:   filebeat/input/journald/testdata/input-multiline-parser-events.json
	new file:   filebeat/input/journald/testdata/input-multiline-parser.export
	modified:   filebeat/input/journald/testdata/input-multiline-parser.journal.gz
	modified:   filebeat/input/journald/testdata/journal1.journal.gz
	modified:   filebeat/input/journald/testdata/journal2.journal.gz
	modified:   filebeat/input/journald/testdata/journal3.journal.gz
	new file:   filebeat/input/journald/testdata/matchers.export
	modified:   filebeat/input/journald/testdata/matchers.journal.gz
	modified:   filebeat/input/journald/testdata/multiple-boots.export
	modified:   filebeat/input/journald/testdata/multiple-boots.journal.gz
	modified:   filebeat/input/journald/testdata/ndjson-parser.journal.gz
	modified:   filebeat/tests/integration/journald_test.go
	new file:   filebeat/tests/integration/testdata/filebeat_journald_all_boots.yml

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   filebeat/input/journald/pkg/journalctl/journalctl.go
	both modified:   filebeat/input/journald/pkg/journalctl/reader_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

• #49526 [9.2](backport #49445) Fix journald input for journalctl versions < 242 by omitting --boot all has been created for branch 9.2 but encountered conflicts

Cherry-pick of f6662a9 has failed:

On branch mergify/bp/9.2/pr-49445
Your branch is up to date with 'origin/9.2'.

You are currently cherry-picking commit f6662a962.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   changelog/fragments/1772726727-fix-journald-args.yaml
	modified:   filebeat/input/journald/README.md
	new file:   filebeat/input/journald/Vagrantfile
	modified:   filebeat/input/journald/input.go
	modified:   filebeat/input/journald/input_filtering_test.go
	new file:   filebeat/input/journald/journald.conf
	modified:   filebeat/input/journald/pkg/journalctl/reader.go
	modified:   filebeat/input/journald/testdata/binary.journal.gz
	new file:   filebeat/input/journald/testdata/convert_export_to_journal.sh
	modified:   filebeat/input/journald/testdata/input-multiline-parser-events.json
	new file:   filebeat/input/journald/testdata/input-multiline-parser.export
	modified:   filebeat/input/journald/testdata/input-multiline-parser.journal.gz
	modified:   filebeat/input/journald/testdata/journal1.journal.gz
	modified:   filebeat/input/journald/testdata/journal2.journal.gz
	modified:   filebeat/input/journald/testdata/journal3.journal.gz
	new file:   filebeat/input/journald/testdata/matchers.export
	modified:   filebeat/input/journald/testdata/matchers.journal.gz
	modified:   filebeat/input/journald/testdata/multiple-boots.export
	modified:   filebeat/input/journald/testdata/multiple-boots.journal.gz
	modified:   filebeat/input/journald/testdata/ndjson-parser.journal.gz
	modified:   filebeat/tests/integration/journald_test.go
	new file:   filebeat/tests/integration/testdata/filebeat_journald_all_boots.yml

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   filebeat/input/journald/pkg/journalctl/journalctl.go
	both modified:   filebeat/input/journald/pkg/journalctl/reader_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

• #49527 [9.3](backport #49445) Fix journald input for journalctl versions < 242 by omitting --boot all has been created for branch 9.3