Qwen3.8-27B-DSpine-b16

---

license: apache-2.0 base_model:

• Qwen/Qwen3.8-27B pipeline_tag: text-generation inference: false tags:
• dspine
• speculative-decoding
• draft-model
• sglang

---

DSpine generates draft tokens in parallel. Each layer passes predicted predecessor features to adjacent positions, carrying causal dependencies through network depth.

This checkpoint is a five-layer draft model for Qwen3.8-27B, with block size 16 and an 8,192-token sliding window.

Evaluation

• Runtime: SGLang 0.5.22, BF16, FlashAttention 3, concurrency 8.
• Draft blocks: DSpine 16; official DFlash2 8.
• DSpine inference: dynamic verification referencing DSpark, with ReplaySSM enabled.
• General benchmarks: xhigh reasoning, greedy and temperature sampling, at most 4,096 new tokens.
• Agent benchmarks: temperature sampling with low and xhigh reasoning; 20 conversations per subset, all recorded tool-decision prefixes replayed in order, prefix caching enabled, at most 32,768 new tokens per request.
• Temperature sampling: temperature=1.0, top_p=0.95, top_k=20.

Results

General Benchmark Acceptance Length

Task	DFlash2(T=0)	DSpine(T=0)	DFlash2(T=1)	DSpine(T=1)
GSM8K	5.69	7.20	5.45	6.31
MATH-500	5.49	6.49	5.27	5.66
HumanEval	4.87	5.71	4.34	4.71
MBPP	5.04	6.14	4.79	5.41
MT-Bench	4.11	4.48	3.89	4.03

Agent Acceptance Length

Agent End-to-End Throughput

Throughput is total output tokens divided by end-to-end concurrent replay wall time (tokens/s). Parentheses show speedup over autoregressive decoding (AR).

Installation

git clone https://github.com/Tencent/DSpine.git
cd DSpine
pip install -e .

Usage

sglang serve \
  --model-path Qwen/Qwen3.8-27B \
  --speculative-algorithm DSPINE \
  --speculative-draft-model-path /path/to/Qwen3.8-27B-DSpine-b16 \
  --speculative-num-draft-tokens 16 \
  --dtype bfloat16 \
  --attention-backend fa3 \
  --speculative-draft-attention-backend fa3 \
  --linear-attn-backend triton \
  --enable-linear-replayssm-spec \
  --mamba-ssm-dtype float32 \
  --mamba-radix-cache-strategy extra_buffer \
  --max-mamba-cache-size 64 \
  --mem-fraction-static 0.80 \
  --context-length 131072 \
  --max-running-requests 8 \
  --cuda-graph-max-bs-decode 8 \
  --min-free-slots-delay 1 \
  --sampling-backend flashinfer \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --trust-remote-code

Acknowledgements

We thank DFlash, SpecForge, SGLang and FlashInfer. Dynamic verification references DSpark.

Citation

@article{zhang2026dspine,
  title   = {Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding},
  author  = {Zhang, Haohui and Chen, Keyu and Sun, Haocheng and Gu, Weibo and Qiao, Ruizhi and Sun, Xing and Jiang, Bo},
  journal = {arXiv preprint arXiv:2609.36173},
  year    = {2026}
}

License

Apache 2.0