CLDSRV-864: add an error listener to TrailingChecksumTransform by leif-scality · Pull Request #6092 · scality/cloudserver

leif-scality · 2026-03-05T14:06:05Z

Cloudserver crashes if TrailingChecksumTransform encounters an error because an exception is thrown. The fix is to add a error listener to the TrailingChecksumTransform.

Crash trace:

{"name":"S3","clientIP":"::1","clientPort":44180,"httpMethod":"PUT","httpURL":"/leif-us-east-1/raw_request.txt","bucketName":"leif-us-east-1","objectKey":"raw_request.txt","bodyLength":0,"bytesReceived":15,"service":"s3","action":"PutObject","accountDisplayName":"Bart","accountName":"Bart","time":1772711134918,"req_id":"d7b458e3494dbb61f876","level":"error","message":"stream ended without closing chunked encoding","hostname":"xps","pid":13612}
2026-03-05T11:45:34.919Z: uncaughtException:
Error: InvalidArgument
    at Object.get (/app/node_modules/arsenal/build/lib/errors/index.js:188:31)
    at TrailingChecksumTransform._flush (/app/lib/auth/streamingV4/trailingChecksumTransform.js:35:36)
    at TrailingChecksumTransform.final [as _final] (node:internal/streams/transform:128:10)
    at prefinish (node:internal/streams/writable:916:14)
    at finishMaybe (node:internal/streams/writable:930:5)
    at Writable.end (node:internal/streams/writable:845:5)
    at IncomingMessage.onend (node:internal/streams/readable:948:10)
    at Object.onceWrapper (node:events:632:28)
    at IncomingMessage.emit (node:events:518:28)
    at IncomingMessage.emit (node:domain:489:12)
{"name":"S3","time":1772711134919,"error":"InvalidArgument","stack":"Error: InvalidArgument\n    at Object.get (/app/node_modules/arsenal/build/lib/errors/index.js:188:31)\n    at TrailingChecksumTransform._flush (/app/lib/auth/streamingV4/trailingChecksumTransform.js:35:36)\n    at TrailingChecksumTransform.final [as _final] (node:internal/streams/transform:128:10)\n    at prefinish (node:internal/streams/writable:916:14)\n    at finishMaybe (node:internal/streams/writable:930:5)\n    at Writable.end (node:internal/streams/writable:845:5)\n    at IncomingMessage.onend (node:internal/streams/readable:948:10)\n    at Object.onceWrapper (node:events:632:28)\n    at IncomingMessage.emit (node:events:518:28)\n    at IncomingMessage.emit (node:domain:489:12)","level":"fatal","message":"caught error","hostname":"xps","pid":13612}

bert-e · 2026-03-05T14:06:09Z

Hello leif-scality,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2026-03-05T14:06:18Z

Incorrect fix version

The Fix Version/s in issue CLDSRV-864 contains:

None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

9.4.0

Please check the Fix Version/s of CLDSRV-864, or the target
branch of this pull request.

claude · 2026-03-05T14:07:37Z

LGTM

Review by Claude Code

codecov · 2026-03-05T14:10:04Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.39%. Comparing base (5bb36b8) to head (684682c).
⚠️ Report is 9 commits behind head on development/9.4.
✅ All tests successful. No failed tests found.

Additional details and impacted files

Files with missing lines	Coverage Δ
lib/api/apiUtils/object/prepareStream.js	`100.00% <100.00%> (ø)`

... and 4 files with indirect coverage changes

@@               Coverage Diff                @@
##           development/9.4    #6092   +/-   ##
================================================
  Coverage            84.38%   84.39%           
================================================
  Files                  206      206           
  Lines                13328    13329    +1     
================================================
+ Hits                 11247    11249    +2     
+ Misses                2081     2080    -1

Flag	Coverage Δ
file-ft-tests	`68.25% <100.00%> (-0.03%)`	⬇️
kmip-ft-tests	`27.94% <0.00%> (-0.01%)`	⬇️
mongo-v0-ft-tests	`69.47% <100.00%> (-0.01%)`	⬇️
mongo-v1-ft-tests	`69.57% <100.00%> (+0.09%)`	⬆️
multiple-backend	`35.11% <0.00%> (-0.01%)`	⬇️
sur-tests	`35.70% <0.00%> (+0.02%)`	⬆️
sur-tests-inflights	`37.51% <0.00%> (-0.04%)`	⬇️
unit	`70.02% <100.00%> (+0.04%)`	⬆️
utapi-v2-tests	`34.41% <0.00%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

anurag4DSB · 2026-03-05T14:36:16Z

lib/api/apiUtils/object/prepareStream.js

    }

    const trailingChecksumTransform = new TrailingChecksumTransform(log);
+    trailingChecksumTransform.on('error', err => errCb(err));


Suggested change

trailingChecksumTransform.on('error', err => errCb(err));

trailingChecksumTransform.on('error', errCb);

This should work

claude · 2026-03-05T15:25:09Z

LGTM

Review by Claude Code

BourgoisMickael · 2026-03-05T15:35:09Z

lib/api/apiUtils/object/prepareStream.js

 }

-function stripTrailingChecksumStream(stream, log) {
+function stripTrailingChecksumStream(stream, log, errCb) {


I see multiple transform streams and pipes. To simplify piping through multiple streams and ensure error propagation and cleanup, we could use stream.pipeline.

…vent unhandled exceptions

claude · 2026-03-06T08:53:46Z

LGTM

Review by Claude Code

bert-e · 2026-03-06T08:53:55Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

leif-scality · 2026-03-06T13:36:31Z

/approve

bert-e · 2026-03-06T13:36:45Z

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

✔️ development/9.4

The following branches will NOT be impacted:

development/7.10
development/7.4
development/7.70
development/8.8
development/9.0
development/9.1
development/9.2
development/9.3

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

Any commit you add on the source branch will trigger a new cycle after the
current queue is merged.
Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

bert-e · 2026-03-06T13:57:58Z

Queue build failed

The corresponding build for the queue failed:

Checkout the status page.
Identify the failing build and review the logs.
If no issue is found, re-run the build.
If an issue is identified, checkout the steps below to remove
the pull request from the queue for further analysis and maybe rebase/merge.

Remove the pull request from the queue

Add a /wait comment on this pull request.
Click on login on the status page.
Go into the manage page.
Find the option called Rebuild the queue and click on it.
Bert-E will loop again on all pull requests to put the valid ones
in the queue again, while skipping the one with the /wait comment.
Wait for the new queue to merge, then merge/rebase your pull request
with the latest changes to then work on a proper fix.
Once the issue is fixed, delete the /wait comment and
follow the usual process to merge the pull request.

bert-e · 2026-03-06T15:10:50Z

I have successfully merged the changeset of this pull request
into targetted development branches:

✔️ development/9.4

The following branches have NOT changed:

development/7.10
development/7.4
development/7.70
development/8.8
development/9.0
development/9.1
development/9.2
development/9.3

Please check the status of the associated issue CLDSRV-864.

Goodbye leif-scality.

leif-scality changed the title ~~CLDSRV-864: add an error listener to TrailingChecksumTransform to pre…~~ CLDSRV-864: add an error listener to TrailingChecksumTransform Mar 5, 2026

anurag4DSB reviewed Mar 5, 2026

View reviewed changes

leif-scality force-pushed the bugfix/CLDSRV-864-crash-TrailingChecksumTransform-unhandled-error branch from 37db0c9 to 137bb7b Compare March 5, 2026 15:24

BourgoisMickael reviewed Mar 5, 2026

View reviewed changes

BourgoisMickael approved these changes Mar 5, 2026

View reviewed changes

anurag4DSB approved these changes Mar 5, 2026

View reviewed changes

CLDSRV-864: add an error listener to TrailingChecksumTransform to pre…

684682c

…vent unhandled exceptions

leif-scality force-pushed the bugfix/CLDSRV-864-crash-TrailingChecksumTransform-unhandled-error branch from 137bb7b to 684682c Compare March 6, 2026 08:52

bert-e merged commit 46d8f75 into development/9.4 Mar 6, 2026
51 of 52 checks passed

bert-e deleted the bugfix/CLDSRV-864-crash-TrailingChecksumTransform-unhandled-error branch March 6, 2026 15:10

	trailingChecksumTransform.on('error', err => errCb(err));
	trailingChecksumTransform.on('error', errCb);

Conversation

leif-scality commented Mar 5, 2026

Uh oh!

bert-e commented Mar 5, 2026

Hello leif-scality,

Uh oh!

bert-e commented Mar 5, 2026

Incorrect fix version

Uh oh!

claude bot commented Mar 5, 2026

Uh oh!

codecov bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

anurag4DSB Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

claude bot commented Mar 5, 2026

Uh oh!

BourgoisMickael Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

claude bot commented Mar 6, 2026

Uh oh!

bert-e commented Mar 6, 2026

Waiting for approval

Uh oh!

leif-scality commented Mar 6, 2026

Uh oh!

bert-e commented Mar 6, 2026

In the queue

Uh oh!

bert-e commented Mar 6, 2026

Queue build failed

Uh oh!

bert-e commented Mar 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Mar 5, 2026 •

edited

Loading