Repository navigation
Conversation
sufubao
force-pushed
the
fix-qwen35-draft-norm
branch
from
October 12, 2026 07:00
a253e1d to
e71c34d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
修复 MTP hidden 的 final norm 约定:Qwen3.5 和 GLM5 Next 在后续 draft 步骤中固定套用 main norm;DeepSeek 的 main/draft、GLM4 MoE Lite 的 main 则漏传了归一化后的 hidden。公共输出层按 FinalHiddenPolicy 枚举准备 NONE、PRE_NORM 或 POST_NORM,norm 与投影分开,logits 和需要传递的归一化 hidden 共用一次计算。普通非 MTP prefill 仍先选最后位置再做 norm。
逐一核对了全部已注册 draft 家族,保留各模型的差异:
Qwen3.5、GLM5 Next 删除 draft 输入端固定的 main norm 调用与绑定。BaseModel 直接收集 PostLayerOutput.final_hidden,漏返回不再回退到原始 hidden;reward head 复用公共准备逻辑,DSpark 直接投影路径显式执行 norm。
参考 vLLM 的 GLM5 Next MTP、DeepSeek MTP、GLM4 MoE Lite main 和 draft。GLM4 MoE Lite draft 按其参考实现保留 PRE_NORM,不随主模型继承 POST_NORM。
MTP 输出约定统一声明在 draft registry 的 MtpPairConfig 中,按 draft checkpoint 的 model_type 与 MTP 模式选择,分别配置 main_final_hidden_policy 和 draft_final_hidden_policy。模型类不再持有 mtp_final_hidden_policy,避免 draft 继承 main 策略。MtpManager 为 main 读取首个 draft 配对,为 draft 读取自身 checkpoint 的配置,并缓存解析结果;非 MTP 返回 NONE,不读取 draft 配置。策略枚举和配对配置位于轻量的 common/mtp_config.py,保持模型注册与 tokenizer 的延迟导入。
BaseModel 将有效策略写入 infer_state.final_hidden_policy,post layer 用当前模型的 norm 权重返回 PostLayerOutput.final_hidden;collector 只收集结果,不持有 policy。autotune 使用 NONE。
验证:540 项相关测试全部通过,覆盖全部注册配对的 main/draft 策略、同一 main 切换配对、多个 draft checkpoint、配置缓存、infer-state 传递、CPU FP32/CUDA BF16 连续两步 draft、post layer、DSpark、CUDA graph、proposer、draft registry 与 tokenizer 导入隔离。已合入最新 main,并补齐 DFlash2 的 NONE/NONE 配对策略;本轮完整选集无跳过、无排除。pre-commit(Black、flake8)和 diff 检查通过。测试通过 exp 留档:261014-151837-python-m-pytest-q-unit-tests-common-basemodel-te。本轮未运行完整模型接受率、吞吐或真实 TP/DP 多卡对照。
#1624 分支已同步至本 PR 最新提交 69ff8c0,在此基础上统一 collector 输出为 ModelMtpOutput.spec_hidden;按要求保持关闭。