arxiv.org·2026/10/20
针对World Action Models的视觉预测目标难以区分任务关键转变与偶然场景变化,Language-Forcing用冻结VLM(Gemma 4 12B)读取当前与未来帧,评估任务进展并预测后续状态,经冻结UMT5编码后对齐动作条件的中间未来视觉特征,仅训练时使用、不改推理路径。在HumanoidArena七项全身任务上平均成功率79.17%,比最强基线GR00T N1.7高9.46个百分点,比同一WAM未加语言对齐高10.77个百分点;真实世界五项任务平均成功率从53.33%升至71.33%。消融表明任务进展语义优于未来图像描述、语言须与观测匹配、更多中间未来帧有助于预测,数据效率在全部演示规模均有提升。部署推理路径不变、无额外计算开销。
为什么重要 对做具身策略的人,这是可插拔的训练期监督,不改推理栈就能在仿真和真机涨点。适合 做具身策略、人形全身控制与 WAM 训练的研究和工程人员
World Action Models(WAMs)联合预测机器人动作和未来观测。然而,它们的视觉预测目标并未显式区分任务相关的转变与偶然的场景变化。对于全身策略而言,这一区分至关重要,因为任务进展依赖于身体姿态、接触和物体状态的协调变化。我们提出了LANGUAGE-FORCING,这是一种训练策略,利用语言来强调全身WAM未来视觉表示中与任务相关的变化。
给定成对的当前帧与真实(ground-truth)的短时程未来帧,一个冻结的视觉-语言模型会评估任务进展并预测合理的后续状态变化。我们使用冻结的文本编码器对这些描述进行编码,并将其嵌入与中间的、以动作为条件的未来视觉特征对齐,从而补充原有的视觉与动作预测目标。所有辅助组件仅在训练期间使用,不会引入任何额外的推理开销。
在HumanoidArena上,LANGUAGE-FORCING取得了79.17%的平均成功率,比最强的竞争策略高出9.46个百分点,比未进行语言对齐的同一WAM高出10.77个百分点。在五个真实世界任务上,语言对齐使同一WAM基线的平均成功率从53.33%提升到71.33%。
World Action Models (WAMs) jointly predict robot actions and future observations.
However, their visual prediction objectives do not explicitly distinguish task-relevant transitions from incidental scene changes.
For whole-body policies, this distinction matters because task progress depends on coordinated changes in body pose, contact, and object state.
We introduce LANGUAGE-FORCING, a training strategy that uses language to emphasize task-relevant changes in the future visual representations of a whole-body WAM.
Given paired current and ground- truth short-horizon future frames, a frozen vision-language model assesses task progress and predicts plausible subsequent state changes.
We encode these descriptions with a frozen text encoder and align their embeddings with intermediate, action-conditioned future-visual features, complementing the original visual and action prediction objectives.
All auxiliary components are used only during training, introducing no additional inference overhead.
On HumanoidArena, LANGUAGE-FORCING achieves an average success rate of 79.17%, exceeding the strongest competing policy by 9.46 percentage points and the same WAM without language alignment by 10.77 percentage points.
Across five real-world tasks, language alignment raises average success from 53.33% to 71.33% over the same WAM baseline.



| 对比项 | 之前 / 基线 | 现在 / 本方案 | |
|---|---|---|---|
| HumanoidArena 平均成功率 | 最强竞争策略 | Language-Forcing 79.17%+9.46 pp | |
| 同架构 WAM 对照 | 无语言对齐 WAM | Language-Forcing+10.77 pp | |
| 五项真机平均成功率 | 同一 WAM 53.33% | 71.33%语言对齐训练 |
EVIDENCE / METHODS
要点
术语解释World Action Model,联合预测动作与未来观测的策略模型 联合预测动作与未来观测,但视觉预测损失不显式区分任务相关变化与视角漂移等无关变化。
原版排版与图表 · 手机无法内嵌时可在浏览器打开。PDF 插件内选文不能传给网页助手。