A reliable, dependency-free C++17 framework for converting model tensors in real time across formats and architectures — safetensors ⇄ GGUF, dtype changes (bf16/fp16/fp32), and the architecture transforms that actually block model interop: MoE expert fuse/unfuse and tensor-name remapping.
It exists because of a concrete failure: two checkpoints of the same 35B model
couldn't be combined — one stored experts fused (…experts.gate_proj shape
[E,I,H], mtp./top-level names), the other unfused
(…experts.{e}.gate_proj shape [I,H], under a model.language_model.
wrapper). Only ~2% of tensor names matched. TensorBridge normalizes both to one
convention so they line up.
codecs (format) canonical adapters (architecture)
┌────────────┐ ┌───────────┐ ┌──────────────────────┐
│safetensors │──read──► │TensorDesc │ ──plan──►│ rename / fuse / unfuse│──►stream out
│ GGUF │ (mmap) │ dtype,shape│ │ (COPY/SLICE/FUSE) │ (safetensors)
└────────────┘ │ byte-span │ └──────────────────────┘
└───────────┘
- Canonical
TensorDesc— name,DType, shape, and a zero-copy[ptr,len)byte span into an mmap. Everything speaks this; nothing copies to translate. - Codecs parse a container's metadata into
TensorDescand stream bytes back out. Reads are mmap'd (bounded RSS — a 65 GB shard costs ~0 resident until touched); writes are two-pass (declare → stream), one tensor resident at a time. - Adapters emit a
TransformPlan: an ordered list of output tensors, each a recipe over the inputs —COPYone,SLICEa sub-tensor out of a fused axis, orFUSEN tensors into one. The executor streams the plan with bounded memory. - dtype engine routes every float conversion through IEEE-754 fp32 with
round-to-nearest-even; integer/bool tensors pass through byte-exact (never
reinterpreted). Adding a float type is one
<->fp32pair.
- Every offset is bounds-checked against real file size; every dense tensor's
byte length is checked against
dtype × shape. Truncated/corrupt shards fail loudly at open (the exact bug that motivated this — a bad download). make test(4 suites): dtype conversions validated bit-for-bit against PyTorch (100.0000% element match, bf16→fp16 on real weights); byte-exact MoE fuse→unfuse→fuse round-trip; GGUF read→write byte-identical round-trip; streaming cast/passthrough/offset-reorder.- Real-file validation: Q4_K/Q6_K dequant matches gguf-py element-for-element;
GGUF writer reproduces a real 4.9 GB / 8B
Q4_K_Mfile byte-for-byte; wire-stream output is byte-identical to batch convert on a 3 GB / 2603-tensor shard. - Portable, zero deps: builds and passes all tests on x86_64 (g++ 12) and aarch64 (g++ 13, DGX Spark) from identical source.
tbridge inspect <file> [--full] # safetensors or GGUF (auto-detect)
tbridge convert <in> <out> [--dtype F16|BF16|F32] # any->safetensors; GGUF quant is dequantized
tbridge arch <in> <out> --op unfuse-experts|fuse-experts [--dtype ..]
tbridge rename <in> <out> --rule 'pat=>rep' [--rule ..] # regex name remap
tbridge stream [--dtype F16|BF16|F32] # safetensors stdin -> stdout (convert on the wire)
tbridge togguf <in.gguf> <out.gguf> # GGUF read -> write (faithful re-encode)
tbridge dumptensor <file> <name> [--n K] # dequant/load a tensor to fp32 (validation)
Convert during a network copy (bounded memory, one tensor resident, no temp file):
tbridge stream --dtype F16 < model.safetensors | ssh host 'cat > model_f16.safetensors'
Example — reconcile the 35B pair, then it can be merged:
tbridge arch fused.safetensors unfused.safetensors --op unfuse-experts
tbridge rename unfused.safetensors normed.safetensors \
--rule 'model\.language_model\.=>model.' --rule 'mtp\.=>model.'
| Format | Read | Write | Notes |
|---|---|---|---|
| safetensors | ✅ | ✅ | mmap + streaming (file or stdin/stdout); F64/F32/F16/BF16, all int/bool |
| GGUF | ✅ | ✅ | header+all-typed-metadata+tensors; float first-class; Q8_0/Q4_0/Q4_K/Q6_K dequantized to float, other quant types opaque passthrough |
make # -> ./tbridge
make test # dtype + adapter + gguf-writer + stream suites
- More quant types (Q5_K/Q3_K/Q2_K/…): add the block layout to
quant.hpp; it plugs into theDTypeblock path and the GGUF→float bridge automatically. - safetensors→GGUF (quantizing) writer: the GGUF writer + typed-KV model are in place; what remains is synthesizing arch metadata and a float→quant encoder.
- New arch adapters: emit a
TransformPlan; the streaming executor is generic.