lib: implement hashset

[pzmarzly]

Currently, bf_set uses bf_list under the hood. This is suboptimal performance-wise. Here I am changing bf_set to be backed by bf_hashset.

• lib: helper: add FNV-1a hash function - imported from another PR, a simple hash function.
• lib: add bf_hashset - main commit. Implements a simple hashset (tombstones on deletion, grows 2x when load factor exceeds 50%, never shrinks, linear probing on collision). See https://en.wikipedia.org/wiki/Linear_probing (with "special flag value" strategy in "Deletion" section).
• lib: set: use bf_hashset for elems field - changes bf_set's inner structure

See #460 for previous attempt.

Fixes #418

[github-actions]

Claude review of PR #470 (d19f0fc)

Must fix

• foreach_break test asserts on unmodified variable — Fixed: count is now incremented before break, so count == 1 is correct.
• No test for bf_hashset_take — Fixed: take test now in tests/unit/libbpfilter/core/hashset.c:272.
• void * pointer arithmetic in bf_vector_foreach will break pedantic C17 integration test (dismissed) — Maintainer explicitly requested not using char * casts; both maintainer and author agreed this is a future topic.
• Uninitialized old_slots in _bf_hashset_grow causes heap corruption on OOM — Fixed: old_slots is now a plain void ** without cleanup attribute (bf_vector removed from PR). Assigned before use.
• Unchecked bf_hashset_add return in _bf_hashset_grow — Fixed: return value is now checked with if (r) { free(old_slots); return r; }. See new suggestion below about the error path's element-loss subtlety.
• Missing <assert.h> in vector.h — Fixed: vector.h was removed from this PR (bf_vector split into a separate PR per reviewer feedback).
• Load factor threshold is 50%, not 70% as described — Fixed: PR description now correctly states 50%, matching the code constants _BF_HASHSET_MAX_LOAD_NUM=5 / _BF_HASHSET_MAX_LOAD_DEN=10.
• bf_hashset_reserve ignores needed when computing new_cap — src/libbpfilter/core/hashset.c:235 — new_cap is derived from set->cap (or _BF_HASHSET_INIT_CAP) instead of from the computed needed value. On an empty set, bf_hashset_reserve(&set, 100) resizes to 16 slots instead of ~256. All 5 independent reviewers flagged this.
• _bf_round_next_power_of_2 missing >> 32 shift for 64-bit size_t — src/libbpfilter/core/hashset.c:32 — Only shifts up to >> 16, so values above 2^32 return non-power-of-two results. Becomes critical once the reserve bug is fixed. All 5 independent reviewers flagged this.

Suggestions

• Tombstone sentinel (void *)1 hardcoded in public macro — Fixed: bf_set_foreach now delegates to bf_hashset_foreach which uses bf_hashset_slot_is_tombstone() function. No literal sentinel in public headers.
• Inconsistent typedef convention between bf_hashset and bf_vector — Fixed: both now use typedef struct bf_* pattern consistently.
• Missing boundary test for bf_vector_new with elem_size == 0 — Fixed: new_zero_elem_size test now in tests/unit/libbpfilter/core/vector.c.
• No test coverage for bf_vector_take and bf_vector_set_len — Fixed: take and set_len tests now in tests/unit/libbpfilter/core/vector.c.
• Commits lack body explaining "why" — The style guide says "Use the commit description to explain why the change is necessary." PR now has 3 commits: commit 1 (lib: helper: add FNV-1a hash function) has a body but it describes "what" more than "why", commits 2 (lib: core: add bf_hashset) and 3 (lib: set: use bf_hashset for elems field) have no body.
• No test for bf_vector_data — Fixed: data test now in tests/unit/libbpfilter/core/vector.c.
• No test for bf_hashset with NULL free callback — tests/unit/libbpfilter/core/hashset.c — All tests use non-NULL .free callback. The if (set->ops.free) branch in bf_hashset_clean is never exercised.
• Redundant forward typedef and non-standard cleanup macro placement in vector.h — Fixed: forward typedef removed; cleanup macros now placed after struct definition.
• bf_set_add_elem_raw performs a redundant allocation and copy — Fixed: bf_set_add_elem_raw now calls bf_hashset_add directly instead of going through bf_set_add_elem, avoiding the extra malloc+memcpy.
• bf_hashset_take doc exposes internal tombstone sentinel value — Fixed: doc now references bf_hashset_slot_is_tombstone instead of the concrete (void *)1 value.
• No test for bf_set_name, bf_set_n_comps, bf_set_key — Fixed: getters test now in tests/unit/libbpfilter/set.c:433-441.
• No test for tombstone accumulation forcing rehash — tests/unit/libbpfilter/core/hashset.c — The remove_and_readd test covers a single remove-readd cycle but does not exercise the scenario where many removals cause tombstone accumulation, increasing slots_in_use until the load factor triggers _bf_hashset_grow. Consider a stress test that inserts, removes many elements, then inserts again to verify the grow-and-compact path.
• bf_set_is_empty not explicitly tested — tests/unit/libbpfilter/set.c — bf_set_is_empty was modified in this PR (switched from bf_list_is_empty to bf_hashset_is_empty) but no test calls it directly. Consider adding assertions in the existing test functions.
• API break: bpfilter/list.h renamed without compat header (dismissed) — The bf_list header rename is no longer part of this PR; it was split into a separate change.
• bf_set_foreach not tested in set unit tests — Moot: bf_set_foreach does not exist in the current diff; callers use bf_hashset_foreach directly on set->elems.
• _bf_hashset_insert_unchecked implicit tombstone-free precondition — Fixed: _bf_hashset_insert_unchecked was removed; _bf_hashset_grow now calls bf_hashset_add directly.
• void * pointer arithmetic in vector.c functions (dismissed) — Maintainer explicitly requested not using char * casts.
• bf_set_add_elem_raw not directly tested — tests/unit/libbpfilter/set.c — This public function is only called indirectly through bf_set_new_from_raw. Its error paths and duplicate-element early return lack direct test coverage.
• _bf_hashset_grow error path leaves hashset in partially-migrated state — Fixed: error path now correctly restores old_slots, cap, len, and slots_in_use before freeing new_slots. No elements are lost on failure. The error path is unreachable in practice (doubled capacity prevents nested grow, no duplicates possible), but the rollback logic is correct.
• FNV-1a test lacks known-answer reference value — Fixed: test now asserts FNV-1a of 'a' equals 0xaf63dc4c8601ec8cULL.
• _clean_bf_hashset_ has no safe default initializer — Fixed: bf_hashset_default() macro now defined at hashset.h:53.
• _bf_hashset_index lacks cap > 0 precondition (dismissed) — Author states asserts are only for NULL-ness checks.
• bf_hashset_slot_is_tombstone could be static inline in the header — Fixed: function removed from public API. Now _bf_hashset_slot_is_tombstone (static) in hashset.c. bf_hashset_foreach uses linked-list traversal and no longer needs tombstone checking.
• bf_hashset_foreach should document iteration safety — Fixed: hashset.h:76 now reads "Do not add or remove elements during iteration."
• Integer overflow in _bf_hashset_needs_grow load-factor check — Fixed: _BF_HASHSET_MAX_ELEMS cap (100M elements) ensures slots_in_use * 10 cannot overflow size_t even on 32-bit platforms. Maximum capacity is ~256M, so 256M * 10 = 2.56B < SIZE_MAX_32.
• Tombstone sentinel (void *)1 not rejected by bf_hashset_add — Fixed: bf_hashset_add now checks if (data == _BF_HASHSET_TOMBSTONE) and returns -EINVAL.
• File-static _bf_set_elem_size makes _bf_program_load_sets_maps not thread-safe — Fixed: global removed. _bf_program_load_sets_maps now uses bf_set_foreach and set->elem_size directly, no qsort needed.
• No test for removing all elements and verifying empty state — Fixed: the foreach_remove test adds 5 elements, removes all during iteration, and asserts bf_hashset_is_empty(&set).
• Unbounded probing loop in bf_hashset_add — Fixed: the slot-search in bf_hashset_add now uses a bounded for (size_t i = 0; i < set->cap; ++i) loop, consistent with _bf_hashset_find.
• Tombstone encoding exposed in public struct field doc — Fixed: bf_hashset_elem.data doc now reads "Data pointer, tombstone, or NULL" without exposing the concrete sentinel value.
• bf_set_add_elem duplicate-silencing behavior not tested — Fixed: add_duplicate test in tests/unit/libbpfilter/set.c inserts the same element twice via bf_set_add_elem and asserts size remains 1.
• bf_hashset_remove dangling pointer after free callback — src/libbpfilter/core/hashset.c — After set->ops.free(&elem->data, ...) runs in bf_hashset_delete, the data parameter may be a dangling pointer if the caller passed elem->data from the slot being removed (as foreach_remove does). The code is correct today because data is not used after the free call, but a future edit could introduce a use-after-free. Consider adding a safety comment.
• bft_set_eq compares insertion order, not set equality — Fixed: renamed to bft_set_eq_ordered, which accurately conveys the positional comparison semantics.
• Double probe in bf_hashset_add — Fixed: _bf_hashset_find now accepts a free_index output parameter, combining the duplicate check and free-slot search into a single probe.
• _bf_hashset_grow rehash performs redundant checks — Fixed: _bf_hashset_resize now iterates the linked list directly and inserts into new_slots without going through bf_hashset_add, avoiding redundant duplicate/load-factor checks.
• Document silent -EEXIST handling in set add functions (dismissed) — Author states the behavior was already present.
• _BF_HASHSET_MAX_ELEMS lacks rationale — Fixed: now _BF_HASHSET_MAX_CAP with a comment explaining it's the largest power-of-two that leaves headroom for load-factor arithmetic without overflowing size_t.
• bf_hashset_reserve overflow in needed computation — Fixed: _BF_HASHSET_MAX_CAP guard (SIZE_MAX / 16 + 1) ensures count * _BF_HASHSET_MAX_LOAD_DEN cannot overflow size_t.
• bf_hashset_elem Doxygen says "slot" but it's an element node — Fixed: @brief now reads "Element node stored in a hashset, maintaining an insertion-order linked list."
• free_idx uninitialized in bf_hashset_add — src/libbpfilter/core/hashset.c:249 — free_idx is declared without an initializer and the second _bf_hashset_find call (after grow) discards its return value. Safe today but fragile; initializing free_idx and checking the second find would make the contract explicit.
• _bf_hashset_resize rehash loop lacks documented invariant — src/libbpfilter/core/hashset.c:67 — The while (new_slots[idx]) loop assumes len < new_cap (guaranteed by the 50% load factor). An assert(set->len < new_cap) or comment would make the precondition explicit.
• ABI break: struct bf_set.elems changed from bf_list to bf_hashset — src/libbpfilter/include/bpfilter/set.h:57 — The struct layout changes in a public header. If the library does not yet guarantee ABI stability, this is acceptable but should be noted in the commit message.
• _bf_round_next_power_of_2 duplicated between hashset.c and program.c — src/libbpfilter/core/hashset.c:25 — Identical implementation in both files (and both are missing the >> 32 shift). Consider extracting into helper.h as a shared static inline function.

Nits

• Commit 1 body describes "what" not "why" — Body is "A simple data hashing function. C stdlib lacks one." which restates the title. Should explain why FNV-1a was chosen or why a hash function is needed.
• Commit 4 scope mismatch — Commit lib: set: use bf_hashset for elems field modifies files under src/bfcli/, src/libbpfilter/cgen/, and tests/ but uses the lib: component prefix. Project history uses comma-separated prefixes when multiple components are touched.
• Use bitmask instead of modulo for power-of-two capacity (dismissed) — Author prefers keeping % to avoid breakage if capacity stops being power-of-two.
• Three getter declarations packed without blank line separators — Fixed: each getter now has a full Doxygen docblock acting as separator.
• bf_hashset_foreach cannot be nested — Fixed: current implementation uses only the user-provided elem_var name with no hidden internal variables, so nesting with different variable names works correctly.
• core/vector.* entries not grouped with other core/ entries in CMakeLists.txt — Fixed: all core/ entries now grouped together in both src/libbpfilter/CMakeLists.txt and tests/unit/CMakeLists.txt.
• Commit 2 scope mismatch (dismissed) — The lib: core: move bf_list into subfolder commit is no longer part of this PR.
• Markdown bold in Doxygen — Fixed: **not** no longer appears in the current code.
• Line length in vector.h comment — Fixed: no line exceeds 80 characters.
• Commit 4 title uses vague "some getters" — Fixed: title is now "lib: set: use bf_hashset for elems field".
• Unnecessary (void **) cast on calloc — Fixed: the (void **) cast on calloc in _bf_hashset_grow has been replaced with (bf_hashset_elem *).
• Mid-function variable declaration in _bf_hashset_grow — Fixed: all variables now declared at top of function.
• _bf_hashset_slot_is_live takes void * instead of const void * — Fixed: now takes const void *.
• bf_hashset_ops and bf_hashset structs lack @brief Doxygen — Fixed: both now have @brief doc blocks.
• bf_hashset_init doc says "stack-allocated" — Fixed: now reads "Initialise a hashset in place".
• BF_FNV1A_PRIME exposed in public header (dismissed) — Author wants constants together.
• Copyright year missing in new files (dismissed) — Project moving to yearless model.
• bf_fnv1a with len = 0 edge case not tested — Fixed: test at helper.c:371 now asserts bf_fnv1a(&val_a, 0, BF_FNV1A_INIT) == BF_FNV1A_INIT.
• Mid-block variable declaration in fnv1a_hash test — Fixed: all variables are now declared at the top of the block before any executable statements.
• Missing explicit #include <errno.h> in hashset.c — Fixed: #include <errno.h> is now present at hashset.c:8.
• bf_hashset_default macro missing Doxygen — Fixed: the macro now has a full @brief, @param, and @return Doxygen block.
• Commit 3 scope mismatch — Commit lib: core: add bf_hashset also modifies tests/unit/libbpfilter/core/hashset.c, tests/unit/CMakeLists.txt, and .clang-format. Per project convention, the prefix should include all affected components.
• Multi-line comment uses // style — Fixed: load factor comment at hashset.c:16 now uses /* */ style.
• _dump_n naming convention — Fixed: variable renamed to dump_idx.
• bf_hashset_elem.data doc incorrectly mentions tombstone and NULL — Fixed: inline comment now reads "User-provided data pointer."
• Unnecessary calloc cast in _bf_hashset_resize — src/libbpfilter/core/hashset.c:61 — calloc return is cast to (bf_hashset_elem **), but the other allocation site in this file (bf_hashset_new line 146) and the wider codebase never cast malloc/calloc returns.
• _bf_hashset_resize missing assert(set) — Fixed: assert(set) now present at hashset.c:55.
• bf_hashset_foreach doc should specify bf_hashset_delete — src/libbpfilter/include/bpfilter/core/hashset.h:100 — The doc says "Safe to remove the current element during iteration" but this only holds when removal is done via bf_hashset_delete(). Calling free() directly would corrupt the slot array. Consider: "Safe to call bf_hashset_delete() on the current element during iteration."

CLAUDE.md improvements

• Style guide says "Use backticks to reference function, variable, and parameter names" in Doxygen, but @c is used in new code at core/hashset.h (7 occurrences). The existing codebase has ~16 @c uses and ~203 backtick uses — backticks are the dominant convention. Consider adding an explicit note in CLAUDE.md about preferring backticks over @c and @ref for inline code references.
• The codebase is inconsistent about __attribute__((cleanup(...))) vs __attribute__((__cleanup__(...))) — the new core/hashset.h uses the non-underscored form (matching core/list.h), while the majority of other headers use the double-underscore form. Consider documenting the preferred form in CLAUDE.md.

Resolved from prior review

• FNV-1a hash seeded with 0 instead of BF_FNV1A_INIT — Fixed: all calls now use BF_FNV1A_INIT.
• bf_hashset_add ownership semantics ambiguous for duplicates — Fixed: doc now clarifies -EEXIST return and ownership.
• bf_set_dump lost bf_dump_prefix_last call for last element — Fixed: uses _dump_n counter.
• Duplicated slot-liveness check between header and implementation — Resolved: inline removed from header.
• _BF_HASHSET_TOMBSTONE exposed in public header — Resolved: constant moved to .c file; bf_hashset_slot_is_tombstone() function used in macro.
• Memory leak in bf_set_add_elem when bf_hashset_add fails during grow — Re-examined: _cleanup_free_ on _elem handles the failure path correctly. TAKE_PTR runs only on success. No leak.
• Variable declared after executable statement in bf_set_dump — Re-examined: _dump_n is at the top of the function, before assert(). Placement is correct.
• assert() on non-pointer value in bf_vector_new — Re-examined: code uses if (!elem_size) return -EINVAL, not assert. Correct per style guide.
• bf_hashset_init ctx references elem_size before computation — Re-examined: bf_hashset_init just stores the pointer. elem_size is fully computed before any hash/equal calls. Safe.
• Serialization order now hash-table-dependent — Re-examined: pack/unpack is order-independent. bft_set_eq now uses bf_hashset_contains instead of ordered comparison. No regression.
• bf_set_get_key_comp lacks Doxygen — Dismissed: bf_set_key is a trivial getter; style guide permits skipping Doxygen for these.
• Redundant bf_hashset_contains check in bf_set_add_elem — Dismissed: the pre-check avoids an unnecessary malloc + memcpy + free cycle when the element already exists. Intentional optimization.

Workflow run

[qdeslandes]

Alright, first pass of review and a few things to fix. I'll do a second, deeper pass, when those are solved. That being said, it's a very welcome addition! :D

[pzmarzly]

I let Claude search for optimizations overnight, and it proposed a different representation that has much better cache locality - bf_hashset_elem** elems instead of bf_hashset_elem* elems, i.e. each node is individually allocated, like in bf_list. It makes reads ~40% faster on large sets. Big change coming.

[yaakov-stein]

Claude has a few valid nits/suggestions and I have one last comment on the tests. Overall LGTM once those points are taken care of!

[yaakov-stein]

I let Claude search for optimizations overnight, and it proposed a different representation that has much better cache locality - bf_hashset_elem** elems instead of bf_hashset_elem* elems, i.e. each node is individually allocated, like in bf_list. It makes reads ~40% faster on large sets. Big change coming.

I'm not necessarily opposed to the change as it simplifies some parts of the code, but I'm confused by the claim that this has better cache locality - shouldn't the cache locality here be much worse? Whenever we need to check for equality we need to load a non-contiguous piece of memory. We also can't take advantage of the spatial locality anymore. Can you explain to me what cases you saw ~40% speedup on?

[pzmarzly]

I'm not necessarily opposed to the change as it simplifies some parts of the code, but I'm confused by the claim that this has better cache locality - shouldn't the cache locality here be much worse? Whenever we need to check for equality we need to load a non-contiguous piece of memory. We also can't take advantage of the spatial locality anymore. Can you explain to me what cases you saw ~40% speedup on?

This was surprising as well to me, so you're right, I should have explained.

The benchmark was (a) generate and insert 1 million random IPs, (b) use bf_hashset_foreach to read it back. bf_hashset_foreach is probably the most important function/macro to optimize, as it's used in many heavy operations (serialization, cgen, bfcli printing).

In both versions (I'll call them bf_hashset_elem[] and bf_hashset_elem*[]), foreach requires 2 pointer dereferences to get to the data: first you resolve elem = *elem.next, then read *elem.data.

In bf_hashset_elem*[], elems are malloced in the same order as they are created. This means that elements created later will usually have higher addresses, unless there was something freed in the meantime. This turns out great for reading.

Meanwhile in bf_hashset_elem[] version, elems are ordered in memory by their hash. Following elem = *elem.next often requires going backwards, to memory that wasn't prefetched into cache. Hence the 40% slower results.

With bf_hashset_elem*[], bf_hashset_contains goes from 1 dereference to 2. That is unfortunate. But I think faster foreach is worth it.

[pzmarzly]

Massive thanks for your continued reviews @yaakov-stein . I'm uploading the latest version with small changes. If they look good, I'll reopen this PR, copying the few comments that are still relevant. It was an experiment to me how much I can rely on the PR review bot, turns out it's pretty good, but GH UX starts degrading massively after 50+ comments.

[yaakov-stein]

The benchmark was (a) generate and insert 1 million random IPs, (b) use bf_hashset_foreach to read it back. bf_hashset_foreach is probably the most important function/macro to optimize, as it's used in many heavy operations (serialization, cgen, bfcli printing).

In both versions (I'll call them bf_hashset_elem[] and bf_hashset_elem*[]), foreach requires 2 pointer dereferences to get to the data: first you resolve elem = *elem.next, then read *elem.data.

In bf_hashset_elem*[], elems are malloced in the same order as they are created. This means that elements created later will usually have higher addresses, unless there was something freed in the meantime. This turns out great for reading.

Meanwhile in bf_hashset_elem[] version, elems are ordered in memory by their hash. Following elem = *elem.next often requires going backwards, to memory that wasn't prefetched into cache. Hence the 40% slower results.

With bf_hashset_elem*[], bf_hashset_contains goes from 1 dereference to 2. That is unfortunate. But I think faster foreach is worth it.

Hm, that makes sense. I think the right approach here would be to call out this behavior and mention the tradeoffs in a PR note for Quentin. To me, it seems surprising that we'd optimize a non-classic hashset operation (foreach) at the expense of the classic hashset operations (contains, add, delete). I'd personally expect that a hashset would be optimized for the standard hashset operations, but I understand where you're coming from. If you split this out to a new PR, maybe let's include a note on how and why this decision was made.

[github-actions]

Claude: must-fix: bf_hashset_reserve computes needed from count but then derives new_cap solely from set->cap (or _BF_HASHSET_INIT_CAP), ignoring needed entirely. For example, bf_hashset_reserve(&set, 100) on an empty set computes needed = 200 but sets new_cap = _bf_round_next_power_of_2(16) = 16, resizing to only 16 slots. The unit test assert_true(bf_hashset_cap(&set) >= 200) would catch this at runtime.

new_cap should be derived from needed, e.g.:

new_cap = _bf_round_next_power_of_2(needed);

[github-actions]

Claude: must-fix: _bf_round_next_power_of_2 only shifts up to >> 16, which covers 32-bit values but not 64-bit size_t. For any input where the highest set bit is at position 32 or above, the function returns a non-power-of-two result. This matters because _BF_HASHSET_MAX_CAP is SIZE_MAX / 16 + 1 (well above 2^32 on 64-bit), and once the bf_hashset_reserve bug above is fixed, needed values above 2^32 would produce corrupt table capacities.

Add value |= value >> 32; after this line. The same pre-existing issue exists in cgen/program.c:66.

[github-actions]

Claude: suggestion: _bf_round_next_power_of_2 is defined identically in both hashset.c and cgen/program.c:66 (and both are missing the >> 32 shift). Consider extracting it into helper.h as a shared static inline function so the fix is applied in one place and the copies don't diverge.