---

license: apache-2.0 base_model:

• tencent/Qwen3.8-27B-DSpine-b16 base_model_relation: quantized pipeline_tag: text-generation library_name: gguf tags:
• gguf
• llama.cpp
• dspine
• dflash
• speculative-decoding
• draft-model
• qwen3.8

---

Qwen3.8-27B-DSpine-b16 GGUF

GGUF conversions of tencent/Qwen3.8-27B-DSpine-b16, the DSpine speculative-decoding draft model for Qwen3.8-27B, for use with llama.cpp.

On a single RTX 3090 with a Q4_K_M Qwen3.8-27B target, it runs at ~125 tok/s on code and math and ~90 tok/s on a mixed set of chat tasks. That compares with ~52 tok/s for DFlash2 and ~28 tok/s without speculative decoding (3.2x overall, up to 4.5x on code). See Benchmarks for speed at increasing context depth and the maximum context on 24 GB.

The speedup depends on your GPU and backend. Community results from AMD, Intel and DGX Spark are in Other GPUs and tuning, so check it before you start.

[!IMPORTANT] Upstream llama.cpp does not support DSpine yet. These files need the dspine branch of this fork: https://github.com/Ajay9o9/llama.cpp/tree/dspine

The branch is current llama.cpp master plus DSpine support. Stock llama.cpp refuses to load these files rather than running them incorrectly.

Files

File	Size	Notes
Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf	1.26 GB	Recommended. The code-bridge tensors stay in BF16.
Qwen3.8-27B-DSpine-b16-Q8_0.gguf	2.20 GB	Same acceptance as Q4_K_M, but slightly slower and uses more VRAM.
Qwen3.8-27B-DSpine-b16-BF16.gguf	4.06 GB	Full precision, for re-quantizing.

These files contain the draft model only. You also need a Qwen3.8-27B target GGUF, such as Q4_K_M.

What DSpine is

DSpine extends DFlash. It is a five-layer drafter that reads hidden states from five layers of the target model and drafts a block of 16 tokens in parallel. It adds three things:

• Causal block attention. Each draft position attends only to earlier positions in its block.
• A code bridge after every layer. Each position receives a gated message built from its predecessor's predicted token code. This passes causal dependencies through the depth of the network.
• Serial last-write refinement. For the top 16 candidates at each position, the drafter redoes the last bridge write using each candidate predecessor's exact code. This produces a 16x16 transition table per position. The speculative decoder walks the table from the last accepted token to pick the draft chain, either greedily or by sampling.

See the paper and the original model card for details.

Build

git clone -b dspine https://github.com/Ajay9o9/llama.cpp.git
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j

Run

./build/bin/llama-server \
  -m  Qwen3.8-27B-Q4_K_M.gguf \
  -md Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 15 \
  -ngl 999 --spec-draft-ngl 999 \
  -fa on --jinja -c 32768 -np 1 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --spec-draft-type-k q8_0 --spec-draft-type-v q8_0

• Set --spec-draft-n-max yourself. The default is 3, which uses only a fraction of the 16-token block. 15 is the best value on NVIDIA CUDA, but on some GPUs a smaller value is faster. See Other GPUs and tuning.
• Set -c yourself instead of relying on auto-fit. Auto-fit does not leave enough room for the drafter, and with a vision projector (--mmproj) loaded the first image request can crash. If the log shows failed to allocate ... using host copies, lower -c.
• --spec-draft-sampling probabilistic is optional. It samples the draft chain from the transition table, and helps slightly at temperature > 0.
• Context on a 24 GB card: with a Q4_K_M target and q8_0 KV cache, DSpine at 15 drafted tokens fits up to -c 65536. For more context, lower --spec-draft-n-max (at 7 it fits up to -c 98304), use a smaller target quant, or use a larger GPU. See Maximum context.
• On 24 GB cards with -np 2, use --spec-draft-n-max 7. Two slots with 15-token drafts do not fit.

Other GPUs and tuning

How fast DSpine is depends on how quickly your GPU can verify a block of 16 tokens at once. On an RTX 3090 that step is cheap, so long drafts pay off. On GPUs where small batches are slow, a 15-token draft can cost more than it saves.

[!NOTE] Community results. Except for the RTX 3090 row, these numbers were measured and shared by community members on their own hardware, not by me. Each one used its own prompts, settings and baseline (some compare against no speculation, some against MTP or DFlash2), so treat the rows as a guide rather than a direct comparison. Each row links to the full report. Thank you to everyone who tested!

GPU	Backend	Best n-max	Result	Reported by
RTX 3090, 24 GB	CUDA	15	90 tok/s on mixed tasks, 3.2x without speculation; 129 tok/s on code	Me, benchmarks below
DGX Spark (GB10), 128 GB unified	CUDA	15	Code 76.7 tok/s vs 61.3 for DFlash2. DFlash2 is faster on math, prose and summaries	Community: Alpaczyk (#2)
Radeon AI PRO R9700, 32 GB	ROCm / HIP	15 (7 was slower)	Code about 2.2x faster than the target's MTP, tool calls 3.4x. Prose, chat and images 4-23% slower than MTP	Community: menace1000 (#3)
Intel Arc Pro B65, 32 GB	SYCL	15 (3 for prose)	Code 116.4 tok/s (5.5x without speculation), tool-call JSON 124.3 (5.9x), math 74.0 (3.5x). Prose is best at 3: 36.2 (1.7x)	Community: ashokkumards (#4)
RX 7800 XT, 16 GB	Vulkan (RADV)	2	47.4 tok/s vs 30.9 without speculation (1.53x). n-max 15 dropped to 19.1	Community: howdya (#1)

What to do:

• Sweep --spec-draft-n-max on your own prompts: try 2, 3, 4, 7 and 15. Starting points:
  • NVIDIA (CUDA), AMD on ROCm / HIP and Intel Arc on SYCL: 15.
  • AMD on Vulkan: 2 or 3.
  • Mostly prose or chat: try 3. Draft acceptance on prose is low, so long drafts are mostly rejected.
• DSpine is strongest on code, math and tool calls. For prose, chat and image-heavy requests, DFlash2 or the target's built-in MTP (--spec-type draft-mtp --spec-draft-n-max 2, needs a target GGUF that includes the MTP layer) can be faster, especially on slower GPUs.
• Multiple GPUs: a two-device layer split (the default) was tested and gives the same output as one GPU. Qwen3.8 keeps a copy of its recurrent state for every drafted token, so if one card runs out of memory, lower --spec-draft-n-max or -c.
• Output can differ from runs without a drafter, even at temperature 0. Verification processes several tokens in one batch, which rounds slightly differently from one token at a time, so near-tied tokens can flip. Code and tool calls usually match; prose and image descriptions can diverge.

If you test on other hardware, please open a discussion with your numbers.

Benchmarks

All numbers were measured on the setup below:

GPU	RTX 3090, 24 GB (all layers on GPU, --fit off)
CPU	Ryzen 9 9900X
llama.cpp	dspine branch of Ajay9o9/llama.cpp, CUDA build
Target	Qwen3.8-27B-Q4_K_M.gguf (17.1 GB)
KV cache	q8_0 for both target and drafter, -fa on
Slots	-np 1
Draft sizes	DSpine --spec-draft-n-max 15, DFlash2 --spec-draft-n-max 7 (each drafter's block size minus one)

Maximum context on 24 GB

For each setup, this is the largest -c that loads entirely on the GPU and serves requests with the q8_0 KV cache.

Setup	Max -c
No speculative decoding	176,128
DFlash2 Q4_K_M, 7 drafted	102,400
DSpine Q4_K_M, 15 drafted	65,536
DSpine Q4_K_M, 7 drafted	98,304
DSpine Q8_0, 15 drafted	40,960

The limit comes from the target, not the drafter. Both drafters use a sliding-window KV cache (8k for DSpine), so their own cache stays small. Qwen3.8 is a hybrid model with recurrent layers, though. While it verifies a draft, it keeps a copy of its recurrent state for every drafted token, which comes to about 150 MB per copy. A 16-token DSpine block therefore costs about 1.2 GB more than an 8-token DFlash2 block, and that VRAM comes out of the context budget.

Speed at increasing context depth

These numbers come from llama-benchy, with --pp 0 --tg 256 --runs 1. The context is filled to each depth, and the table shows decode speed for 256 new tokens, in tok/s. Each setup ran with -c set to its maximum, and the last row for each setup is just below that maximum. A dash means the depth does not fit for that setup on 24 GB.

Default corpus (a Sherlock Holmes novel; the model writes prose)

Depth	No speculation	DFlash2 (7)	DSpine Q4_K_M (15)
0	29.5	31.6	48.0
8,192	28.1	26.6	35.6
16,384	26.2	26.1	35.5
32,768	23.8	28.0	31.1
63,488	-	-	35.5
65,536	19.5	23.0	-
100,352	-	25.9	-
131,072	14.6	-	-
174,080	12.5	-	-

Code corpus (llama.cpp C++ sources as context; the model mostly explains the code)

Depth	No speculation	DFlash2 (7)	DSpine Q4_K_M (15)
0	28.0	32.9	41.6
8,192	27.4	28.4	35.2
16,384	25.6	25.4	32.8
32,768	22.8	26.6	29.5
63,488	-	-	47.8
65,536	19.1	22.3	-
100,352	-	24.0	-
131,072	14.5	-	-
174,080	12.2	-	-

Prefill speed for the default corpus, in tok/s:

Depth	No speculation	DFlash2 (7)	DSpine Q4_K_M (15)
8,192	1,115	1,092	969
32,768	975	910	859
65,536	835	795	741 (at 63,488)
131,072	645	-	-

Each cell is a single run of 256 tokens. Prose and code explanations are the hardest content to draft, so expect noise of a few tok/s. On these corpora the drafters average only about 2.5-3.5 tokens per verify step.

Speed by task (short context)

This test used 10 chat prompts, 2 each of math, writing and summarize/translate, plus 4 code prompts in Python, C++, JavaScript and SQL. Each prompt generated up to 1,024 tokens with thinking disabled, about 6,000 tokens per setup. Temperature 1 used top_p 0.95, top_k 20 with probabilistic draft sampling.

Setup	Temperature 0 tok/s	Speedup	Temperature 1 tok/s	Speedup
No speculative decoding	28.2	1.00x	28.2	1.00x
DFlash2 Q4_K_M, 7 drafted	51.7	1.83x	50.6	1.79x
DSpine Q4_K_M, 15 drafted	90.1	3.19x	80.4	2.85x
DSpine Q8_0, 15 drafted	86.3	3.06x	-	-

DSpine Q4_K_M at temperature 0, by task:

Task	tok/s	Tokens per verify step	DFlash2 tok/s
Code	128.7	7.6	63.3
Math	122.8	7.3	64.2
Summarize / translate	62.0	3.6	43.0
Prose / story	41.6	2.5	30.2

Summary: On the RTX 3090, DSpine roughly doubles DFlash2's gain on code and math at short context, and it is still the fastest at every depth it fits. Its trade-off on a 24 GB card is context: about 64k at 15 drafted tokens, compared with about 100k for DFlash2 and 176k without speculation.

Convert it yourself

From the dspine branch:

python convert_hf_to_gguf.py /path/to/Qwen3.8-27B-DSpine-b16 \
  --target-model-dir /path/to/Qwen3.8-27B \
  --outtype bf16 --outfile Qwen3.8-27B-DSpine-b16-BF16.gguf

./build/bin/llama-quantize Qwen3.8-27B-DSpine-b16-BF16.gguf Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf Q4_K_M

--target-model-dir must point to the target model's config.json and tokenizer files. The target's weights are not needed.

License and credit

The weights are from tencent/Qwen3.8-27B-DSpine-b16, released under Apache-2.0. All credit for the model goes to the DSpine authors.

@article{zhang2026dspine,
  title   = {Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding},
  author  = {Zhang, Haohui and Chen, Keyu and Sun, Haocheng and Gu, Weibo and Qiao, Ruizhi and Sun, Xing and Jiang, Bo},
  journal = {arXiv preprint arXiv:2609.36173},
  year    = {2026}
}