数据源与状态仍在导航菜单中
arxiv.org·2026/10/23
针对长 CoT 中 speculative decoding 接受率是否随位置衰减的问题,作者以 DeepSeek-R1-Distill-Qwen-7B 为目标、未用目标输出训练的同族与跨族独立 draft,在 MATH-500 全量题目上用 greedy 与 Leviathan sampling 报告约 1.78M 验证轮。剔除循环样本并做题内配对后,有效位置区间内接受率不随位置下降,后半段反而略高,同族 greedy 后半段比前半段高约 0.03,跨族增幅更大。温度扫描表明,reflection token 的 argmax agreement 下降只是可见性伪影,α 未下降,拒绝集中于高熵反思决策点。LoRA 轨迹对齐未提升 MAL,提示应直接优化接受准则而非拟合轨迹;局限是单域、固定目标族与草稿设置,且未直接验证训练长度失配。
为什么重要 推翻「长 CoT 上 SD 接受率随位置衰减」的部署叙事:其证据全部来自训练型 drafter;做推理加速与草稿训练选型时不应引入位置补偿,应瞄准接受准则。适合 投机解码/推理加速工程师、草稿模型与对齐训练研究者
关于 speculative-decoding acceptance rate 会随着 chain-of-thought 变长而衰减的说法,已成为事实上的行业叙事,并推动了 draft-model co-design 和 position-aware training。几乎所有直接证据都来自训练得到的 drafters,其离线训练长度远短于推理,因此一个纯粹的位置规律从未被分离出来。
我们提出了首次系统、全长测量,将 acceptance rate 作为生成位置的函数,在 MATH-500(全部 500 个问题)上、在 greedy 和 sampling 解码下进行,并使用两个独立 drafts 对比 DeepSeek-R1-Distill-Qwen-7B:一个同家族 draft(DeepSeek-R1-Distill-Qwen-1.5B)和一个跨家族 draft(Qwen2.5-1.5B-Instruct),产生约 1.78M 轮验证。使用问题内配对测试,我们发现,在分离出 greedy 解码重复病理之后,在支持范围(greedy 0-8K,sampling 0-12K)内,四种配置中的任何一种中,acceptance rate 都不随位置衰减;每条轨迹的后半段略且显著地高于前半段。
受控温度扫描表明,高熵反思 token 在采样模式下的表面退化是 argmax agreement 的可见性伪影,而不是期望接受概率下降。拒绝高精度地集中在反思触发 token 上,如 “wait”、“however” 和 “check”,这是一种熵选择性失败;一个轻量级三臂 LoRA trajectory-alignment 干预(uniform、reflection-marked、entropy-weighted)未能改善平均接受长度。
因此,我们提供了独立 draft-target 对一直缺失的位置基线,并将残余失败定位到决策点熵,而不是位置。
The claim that speculative-decoding acceptance rate decays as a chain-of-thought grows longer has become a de facto industry narrative, motivating draft-model co-design and position-aware training.
Nearly all direct evidence comes from trained drafters whose offline training length is far shorter than inference, so a purely positional law has never been isolated.
We present the first systematic, full-length measurement of acceptance rate as a function of generation position on MATH-500 (all 500 problems), under greedy and sampling decoding, with two independent drafts against DeepSeek-R1-Distill-Qwen-7B: a same-family draft (DeepSeek-R1-Distill-Qwen-1.5B) and a cross-family draft (Qwen2.5-1.5B-Instruct), yielding about 1.78M verification rounds.
Using within-problem paired tests, we find that after isolating a greedy-decoding repetition pathology, acceptance rate does not decay with position in any of the four configurations within the supported ranges (greedy 0-8K, sampling 0-12K); the second half of each trajectory is slightly but significantly higher than the first.
A controlled temperature sweep shows that the apparent sampling-mode degradation of high-entropy reflection tokens is a visibility artifact of argmax agreement, not a drop in expected acceptance probability.
Rejections concentrate with high precision on reflection trigger tokens such as "wait", "however" and "check", an entropy-selective failure that a lightweight three-arm LoRA trajectory-alignment intervention (uniform, reflection-marked, entropy-weighted) fails to improve in mean accepted length.
We thus supply the positional baseline that independent draft-target pairs were missing, and localize the residual failure to decision-point entropy rather than position.
验证轮次
四配置合计
贪心 loop 题
采样 16 题;规则 ≥16,000
跨族半差
xfam/greedy 的 def1
采样 术语解释mean accepted length,每轮验证平均接受的 token 数(含 bonus token),决定 SD 加速比 代价
采样相对贪心的 术语解释mean accepted length,每轮验证平均接受的 token 数(含 bonus token),决定 SD 加速比 成本
注释 · 后半减前半的题内配对差;支持范围贪心 0–8K、采样 0–12K出自 §4, 摘要
注释 · 采样模式同桶 α 为 0.788(同族)、0.444(跨族)出自 §6, Fig 3
EVIDENCE / METHODS
表格
全部为正且极显著;跨族效应约为同族两倍
| 配置 | 指标 | Diff | 95% CI | Up/Down |
|---|---|---|---|---|
| same/greedy | def1 | +0.031 | [+0.026,+0.036] | 327/126 |
| same/sample | def1 | +0.028 | [+0.023,+0.033] | 354/130 |
| xfam/greedy | def1 | +0.069 | [+0.063,+0.076] | 393/61 |
| xfam/sample | def1 | +0.059 | [+0.053,+0.065] | 407/78 |
| same/sample | α | +0.010 | [+0.008,+0.013] | 333/151 |
| xfam/sample | α | +0.030 | [+0.027,+0.034] | 365/120 |
注释 · p 为精确符号检验(最大 8.3×10−17);loop 剔除后池化半差 +0.064→+0.025出自 Table 1
原版排版与图表 · 手机无法内嵌时可在浏览器打开。PDF 插件内选文不能传给网页助手。