r0b0tlab/MiniCPM-V-4.7-35B-A3B-DSpark-EXL3-6.00bpw

MiniCPM-V-4.7-35B-A3B-DSpark EXL3 6.00 bpw (draft)

Summary

EXL3 (ExLlamaV3) quantisation of openbmb/MiniCPM-V-4.7-35B-A3B-DSpark (revision 5facb75c), the DSpark block-diffusion drafter for MiniCPM-V-4.7-35B-A3B, for use with r0b0tlab/MiniCPM-V-4.7-35B-A3B-EXL3-6.08bpw.

• The five-layer body is quantised at 6.00 bpw (uncalibrated EXL3, codebook mul1); the Markov head, hidden-state norm, layer norms and confidence head are the upstream bf16 tensors, byte for byte (sha256 verified, 26 tensors).
• 451 MB on disk in one safetensors file.
• ExLlamaV3 runs DSpark drafting on the minicpm-v4.7-dspark branch of exllamav3-r0b0t (commit 5ddae7d12448). Stock ExLlamaV3 does not register this draft's architecture (Qwen3DSparkModel).

Where the numbers on this card come from

Every figure on this card except the upstream table in Acceptance was measured with ExLlamaV3 on one NVIDIA GB10: the quantisation fidelity with the fork's forward pass on the equivalent development branch described under Limitations, and the drafting figures on the minicpm-v4.7-dspark branch (commit 5ddae7d12448) on the 80-prompt Leg D kit (15 drafts per round). The drafting figures are reproducible with the serving repository at v2.1.1. The fidelity figures were measured when the draft was quantised, and not all of them are yet reproduced in the serving repository. The upstream table is the model authors' own measurement of the bf16 pair, quoted from their model card.

• Greedy streams are identical to plain decoding on 59 of 80 prompts with adaptive drafting. The rest differ at some point; the position and the cause were not measured.
• Every draft is verified by the target model. Outputs still differ from plain decoding on some prompts, as listed above.
• One pass per lane on one GB10 at batch 1.

What this draft is

DSpark is a block-diffusion drafter: instead of proposing tokens one at a time, one forward pass proposes a whole block that the target then verifies. Facts of this checkpoint, from its config.json:

property	value
architecture	Qwen3DSparkModel (5 transformer layers, hidden 2048, 8 attention heads / 2 KV heads, head dim 256, MLP 6144)
parameters (upstream, bf16)	389,223,169
block size	15 (1 anchor + 14 mask positions: 15 draft tokens per forward pass)
target layers tapped	3/11/19/27/35 of the 40-layer target (all full-attention layers)
Markov head	vanilla, rank 256
confidence head	enabled; takes the Markov features as extra input
vocabulary	248,144 (the target's)
mask token id	248076

• Shares the target's embedding and lm_head. There are no embed_tokens or lm_head tensors in this repository; the block is embedded with the target's table (mask positions use the row of token id 248076) and the proposals are scored with the target's output head. The target's embedding and head precision therefore enter the draft's logits.
• Block layout, as implemented by the upstream reference code: the anchor token (the last accepted token) followed by 14 mask positions, non-causal inside the block; the draft attends to the target's hidden states from layers 3/11/19/27/35 (concatenated, projected by fc, normalised by hidden_norm) as cached keys and values.
• The Markov head conditions each proposal on the previous proposed token (rank 256 tables markov_w1, markov_w2 of 248,144 x 256); the confidence head scores each proposed position.

What is quantised and what is not

component	count	stored as	bits per weight	size	share
Markov head tables	2 tensors	bf16, byte-identical to the source	16	0.254 GB	56.3%
MLP projections (gate, up, down)	15 matrices	EXL3 trellis, codebook mul1	6	0.142 GB	31.4%
attention projections (q, k, v, o)	20 matrices	EXL3 trellis, codebook mul1	6	0.039 GB	8.7%
fusion projection (fc)	1 matrix	EXL3 trellis, codebook mul1	6	0.016 GB	3.5%
layer norms, hidden_norm, final norm	12 tensors	bf16, byte-identical to the source	16	0.000 GB	0.0%
attention q/k norms	10 tensors	bf16, byte-identical to the source	16	0.000 GB	0.0%
confidence head	2 tensors	bf16, byte-identical to the source	16	0.000 GB	0.0%

• Quantiser settings: 6.00 bpw trellis, stored with fp16 scales, no calibration data (identity Hessian), codebook mul1. The command is in CONVERT.md.
• 26 of the 26 plain (not trellis-coded) tensors are byte-identical to the source checkpoint (sha256 over the raw bytes): layer norms, hidden_norm, final norm (12), attention q/k norms (10), Markov head tables (2), confidence head (2).
• The confidence head is the upstream one: no calibration (Platt) fold was applied to it.
• Per-tensor layout of the pack: pack_layout.tsv.

Size and memory

item	size
weights in this repository	451 MB (0.42 GiB), one file
of which Markov tables (bf16)	254 MB
of which EXL3 body (trellis + scales)	197 MB
draft KV cache	10,240 B per context token (fp16, full attention in all 5 layers)

Draft KV cache per context token = layers x 2 x KV heads x head dim x 2 bytes = 5 x 2 x 2 x 256 x 2 = 10,240 bytes; the 254 MB of Markov tables are only read for the candidate rows of each proposed position.

Fidelity of the quantised body

Uncalibrated EXL3 changes the weights by a relative L2 error that is nearly uniform across the 36 quantised matrices. The last two columns compare one 15-row block pushed through the quantised body with the same block through the bf16 body (5-layer forward, real target embedding rows, random target hidden-state taps; mean of three random-input runs).

body bpw	trellis bpw	with scales	file size	median rel L2 (matrices)	max rel L2	block output rel L2 vs bf16	block output cosine
8.00	8.0000	8.0123	517 MB	0.00481	0.00512	0.0204	0.99979
6.00 (this repository)	6.0000	6.0123	451 MB	0.01741	0.01779	0.0710	0.99747
5.00	5.0000	5.0123	418 MB	0.03396	0.03435	0.1255	0.99211
4.00	4.0000	4.0123	386 MB	0.06677	0.06727	0.2594	0.96641
3.00	3.0000	3.0123	353 MB	0.13193	0.13253	0.4613	0.89318

The 6.00 bpw row is this repository; the other bit widths were measured in the same sweep and are not published here. Per-matrix errors and the checks run on this pack: verify_report.json (error lists are empty, every verbatim tensor equals the source sha256).

Acceptance

Reported by the model's authors for the bf16 draft with the bf16 target (acceptance length = completion tokens per verification step; natural EOS, max_new_tokens 8192; source: the upstream model card):

domain	accept length
Vision
MMMU	5.098
MathVista	6.126
CharXiv_reasoning_val	5.479
OCRBench	5.407
Text
Math	6.687
Code	7.387
General	3.501

Measured with ExLlamaV3 for this quantised draft with the quantised target (r0b0tlab/MiniCPM-V-4.7-35B-A3B-EXL3-6.08bpw), 15 drafts per round, greedy decoding, one request, tokens per second:

prompts (Leg D kit)	n	plain	drafted, adaptive verification	drafted, fixed 15	accepted drafts per generated token (adaptive)	identical to plain, adaptive	identical to plain, fixed 15
GSM8K-style math	40	60.1	177.8	162.1	0.87	33 of 40	33 of 40
HumanEval-style code	20	59.8	220.5	207.3	0.90	20 of 20	19 of 20
IFEval-style chat	20	59.4	69.0	41.0	0.49	6 of 20	8 of 20
all	80	59.9	161.3	143.1	0.79	59 of 80	60 of 80

Adaptive verification is the default: it verifies as many drafts as pay off under a measured cost model of the round. Fixed 15 drafts is slower than plain on chat.

Limitations

• The body is quantised without calibration; the acceptance measurements on this card use the 6.08 bpw target above.
• Stock ExLlamaV3 cannot load or run this draft. The minicpm-v4.7-dspark branch of exllamav3-r0b0t runs it with the target (commit 5ddae7d12448).
• Quantised with the loader branch's draft support (uncalibrated); the fidelity measurements on this card were taken on an equivalent development branch (library code identical apart from comments and one error-message string); the drafting and generation measurements were taken on the minicpm-v4.7-dspark branch, commit 5ddae7d12448.
• The draft was trained against the bf16 target; a quantised target changes the hidden states the draft consumes and can lower acceptance; the measured table above is the only acceptance claim made for this pack.
• Every draft is verified by the target model, but the output is not bit-identical to plain decoding: 59 of 80 greedy prompts match exactly with adaptive drafting.
• The Markov and confidence heads are bf16 and add 254 MB; they are not part of the bit width quoted above.

How to load

This repository is passed, with the target, to the minicpm-v4.7-dspark branch of exllamav3-r0b0t (commit 5ddae7d12448), which runs the draft. Stock ExLlamaV3 cannot load it. The minicpm-v4.7 branch loads the weights (Config.from_directory, Model.from_config, load) but cannot draft with them. The serving repository at v2.1.1 attaches the draft with ./run.sh --with-draft. Layout:

architecture       Qwen3DSparkModel (5 layers, block 15, taps 3/11/19/27/35 of the 40-layer target)
body tensors       <layer>.<proj>.{trellis,suh,svh,mul1}  (EXL3, codebook mul1)
bf16 tensors       hidden_norm, norm, layers.N.*layernorm, q_norm, k_norm, markov_head.markov_w{1,2}, confidence_head.proj.{weight,bias}
shared with target embed_tokens, lm_head

Files

file	bytes	content
model.safetensors	451,186,749	EXL3 draft weights (single file)
CONVERT.md	2,289	conversion command, settings, hardware, wall-clock
LICENSE	11,358	Apache License 2.0
NOTICE	1,727	attribution and statement of modifications
SWEEP.tsv	698	weight-space and block-output error per body bit width
calibration.md	201	calibration data description
config.json	1,873	model configuration (upstream values plus the converter's quantization_config block)
pack_layout.tsv	3,582	tensor layout of the pack versus the upstream checkpoint
quantization_config.json	48,749	EXL3 storage description of every tensor
verify_report.json	14,279	checks run on this pack (sanitised)
README.md	(this file)	this model card
UPLOAD_MANIFEST.json	-	every file with size and sha256

SHA-256

UPLOAD_MANIFEST.json lists every file with size and sha256. The same digests (verify with sha256sum -c after saving the block as a file):

62aa7de3c56270f35bd615a44e974850044ce39fcd3e3cabc135b0ff68202b83  CONVERT.md
cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30  LICENSE
cb54af2c08bce56ff32b807c01416957b94eec0f2cc3978b34b581cb1d087183  NOTICE
e22437a2fb72c2bf1d7599cbf2dfaaa91c0395b31729c97405d9e212140efa64  SWEEP.tsv
9d34af583de8003675941aa2d81eccd6057ad0870b0aad3977dbccf20c188db4  calibration.md
5d010326a72ddae21d126d8a86bf5ffc829e6550177f141618069e60e3d7a8fb  config.json
728cdd789e20c142f1085e16f8381973f3a7018d4fabcbc776c13b6e77eff964  model.safetensors
5839355dfeba5edf18b9f206bb66acf3ed6d34eac84f0f58e62fe43743d719f3  pack_layout.tsv
28c6e3b9784a7390b15dbccf0f8e7a2ddd9da836a395be6f1089026f0a6714a4  quantization_config.json
3e7a19de614d224329037e1470343548edeac6df0280ffe9f70b7582bd05ed5b  verify_report.json

License

Apache License 2.0, inherited from the base model (openbmb/MiniCPM-V-4.7-35B-A3B-DSpark); see LICENSE and NOTICE for the attribution and the statement of modifications.

Thanks to OpenBMB for the model and the DSpark method, and to turboderp for ExLlamaV3 and the EXL3 format. The limitations and disclaimer of the upstream model card apply.