Replace assert() with runtime check for tensor ndim bounds

[MillaFleurs]

Summary

gguf_get_tensor() validates tensor->ndim against GGUF_TENSOR_MAX_DIM using assert(), which is compiled out in release builds (NDEBUG). A crafted GGUF file with ndim > 8 causes the subsequent dimension-reading loop to write past the end of the fixed-size dim[GGUF_TENSOR_MAX_DIM] array, corrupting adjacent struct members (offset, bsize, num_weights, weights_data) and stack memory beyond the struct.

Details

Current code (gguflib.c, line 275):

tensor->ndim = *num_dim;
assert(tensor->ndim <= GGUF_TENSOR_MAX_DIM);  // NO-OP when NDEBUG is defined

The problem: In release builds, assert() expands to nothing. The unchecked ndim value flows into:

for (uint32_t j = 0; j < tensor->ndim; j++) {
    tensor->dim[j] = *dim;    // OOB write when j >= 8

Relevant spec language (ggml/docs/gguf.md):

"The number of dimensions in the tensor. Currently at most 4, but this may change in the future."

GGUF_TENSOR_MAX_DIM is already set to 8 for future-proofing ("Future-proof: actual limit is 4"). The wire format uses uint32_t for n_dimensions, so a malicious file can declare up to 2^32 dimensions.

Fix

Replace the assert() with a runtime check, following the existing error convention used for invalid tensor types at line 290 (if (*type >= GGUF_TYPE_COUNT) return 0):

if (tensor->ndim > GGUF_TENSOR_MAX_DIM) {
    tensor->name = NULL;
    return 0;
}

This maintains API compatibility — callers already handle return 0 with tensor->name == NULL as an error/termination condition (documented in the function comment at lines 251-254).

Impact

Any application using gguflib to load untrusted GGUF files in release builds is affected. Known downstream consumers include Apple MLX, which vendors gguflib at commit 8fa6eb65.

Test plan

• Build with -DNDEBUG and load a GGUF file with ndim=32 — should return 0 instead of OOB write
• Verify existing valid GGUF files still load correctly
• Verify the assert() removal doesn't break debug builds (the if check is strictly stronger)

🤖 Generated with Claude Code

[MillaFleurs]

Opening after this was found to cause an issue in MLX. MLX pr: ml-explore/mlx#3358