Skip to content

perf(assets): read numeric GGUF metadata in bounded blocks - #181

Open
Afloat16 wants to merge 1 commit into
Edge0-AI:mainfrom
Afloat16:afloat/20261111-next-batched-gguf-arrays
Open

Afloat16 wants to merge 1 commit into
Edge0-AI:mainfrom
Afloat16:afloat/20261111-next-batched-gguf-arrays

Conversation

@Afloat16

Copy link
Copy Markdown

What & why

Numeric GGUF arrays performed one Python read/dispatch per scalar, making vocabulary-sized score/type tables expensive.

Decode numeric arrays in blocks of at most 8192 elements (64 KiB), preserving scalar types, little-endian decoding, empty arrays and truncation errors. String arrays retain their path.

Reference: https://github.com/ggml-org/ggml/blob/cd6d2b75f5d3e4b2f3fe8e8fbc6c8b84ac80e74a/docs/gguf.md

Area

Model asset tools (edge0-infer/tools/assets/).

How was it tested?

python -m pytest -q tools/assets/tests/test_gguf_lite.py

16 tests passed with Python 3.12 and NumPy 2.3.5. A 25000-element uint32 table uses 13 reader calls instead of 25009, with a maximum read of 32768 bytes. The baseline fails the read-count bound. All 11 scalar formats, 8193-element/empty arrays, truncated payloads, existing tensor round-trips and quantization are covered.

All 14 L0 checks passed in a local integration with the other focused fixes. git diff --check passed.

Checklist

  • Full ctest --preset cpu -L "L0|L1" passes locally
  • Added/updated focused regressions and ran the affected target
  • No public interface/configuration change requiring additional documentation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant