Open
Conversation
Problem: It would be convenient to know the size of the data in a val or valref object without decoding/reading the data. However there is no metadata for that. Support an optional size key in 'val' and 'valref' objects. It allows us to know the size of the data without wasting cycles decoding or reading the data.
garlick
reviewed
Sep 16, 2025
Comment on lines
+85
to
+90
| The following OPTIONAL names are supported: | ||
|
|
||
| - *size*, contains the size of the decoded data within a *val* object | ||
| or the total size of data stored in the blobrefs of a *valref* object. | ||
|
|
||
|
|
Member
There was a problem hiding this comment.
This seems like it could be neat.
A thought on adding size to dir and dirref: since every KVS metadata update involves updating all parent metadata back to the root, we could use that fact to keep a cumulative size in dir and dirref objects. That is, a directory would contain the sum of the sizes of files and directories it contains. The root would then contain the total size of the namespace. These sizes could be a little bit misleading due to the automatic de-duplication inherent in the content design, but it still might be useful and it would accurately reflect the size of the subtree if dumped with flux-dump(1).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem: It would be convenient to know the size of the data in a val or valref object without decoding/reading the data. However there is no metadata for that.
Support an optional size key in 'val' and 'valref' objects. It allows us to know the size of the data without wasting cycles decoding or reading the data.
As mentioned in
flux-framework/flux-core#6264
flux-framework/flux-core#6266
it would be convenient to know how much data there is in a KVS entry without reading all of the contents.
Side note, it may be useful to also know the size of each blobref in a
valrefobject, but I will leave that for another possible RFC change when the time arrives.