数据源与状态仍在导航菜单中
arxiv.org·2026/10/14·Mingyu Ouyang / Yiran Wang 等
为什么重要 为游戏与数字控制提供合成环境 RL 方案,用可验证奖励替代难以获取的大规模交互环境,代码开源便于复现。适合 游戏智能体、具身控制、RLVR 后训练与合成环境构建方向的工程师/研究者。
计算机使用智能体已在解读屏幕并执行键盘和鼠标操作方面展现出强大能力。然而,游戏因其视觉丰富的世界和多样的机制而仍然具有挑战性,并为通向通用数字控制提供了要求苛刻的下一步。当前的游戏智能体学习流程通常依赖大规模示范,而这些示范对动态游戏状态的覆盖有限;同时,用于交互式训练的可扩展环境仍然难以获得。我们提出 Player-1,一个通过合成经验在示范之外学习游戏控制的后训练框架。
我们通过相对指针运动、持续输入和复合动作扩展了计算机使用动作空间。对于监督初始化,我们在 6,044 条精选的游戏轨迹上微调计算机使用模型,这些轨迹将录制的动作与任务指令和视觉推理配对。我们构建了 Player-1-Gym,这是十六个围绕游戏中共有的控制需求而设计的合成 2D 和 3D 环境。可编程验证的奖励以及随机化的观测和控制动力学,支持使用 PPO 进行多环境强化学习。
由此得到的 Player-1-9B 和 Player-1-35B-A3B 在 MineDojo-300 上的成功率相比其 SFT 策略分别提高了 27.8 和 17.7 个百分点。Player-1-9B 达到 38.33% 的成功率,而最强评估基线的成功率为 12.63%;在 ViZDoom-10 和 GameWorld 上,它还比 SFT 取得了进一步增益。
进一步分析表明,更广泛的合成环境覆盖可以改善迁移,而在 RL 过程中,组合多个键盘和鼠标输入的复合动作使用会增加。我们证明,从合成经验中学习能显著改善游戏控制,超越监督初始化,并且这些增益可泛化到 RL 训练套件之外。
Computer-use agents have demonstrated strong capabilities in interpreting screens and executing keyboard and mouse actions.
However, games, with their visually rich worlds and diverse mechanics, remain challenging and offer a demanding next step toward general digital control.
Current game-agent learning pipelines often rely on large-scale demonstrations with limited coverage of dynamic game states, while scalable environments for interactive training remain difficult to obtain.
We introduce Player-1, a post-training framework for learning game control beyond demonstrations through synthetic experience.
We extend the computer-use action space with relative pointer motion, sustained inputs, and composite actions.
For supervised initialization, we fine-tune computer-use models on 6,044 curated gameplay trajectories that pair recorded actions with task instructions and visual reasoning.
We construct Player-1-Gym, sixteen synthetic 2D and 3D environments designed around control demands shared across games.
Programmatically verifiable rewards and randomized observations and control dynamics support multi-environment reinforcement learning with PPO.
The resulting Player-1-9B and Player-1-35B-A3B improve MineDojo-300 success over their SFT policies by 27.8 and 17.7 percentage points, respectively.
Player-1-9B reaches 38.33% success compared with 12.63% for the strongest evaluated baseline, with further gains over SFT on ViZDoom-10 and GameWorld.
Further analyses show that broader synthetic environment coverage improves transfer, while the use of composite actions that combine multiple keyboard and mouse inputs increases during RL.
We demonstrate that learning from synthetic experience substantially improves game control beyond supervised initialization, with gains that generalize beyond the RL training suite.



EVIDENCE / METHODS
要点
游戏视觉复杂、机制多样,是 computer-use agents 迈向通用数字控制的下一步。
原版排版与图表 · 手机无法内嵌时可在浏览器打开。PDF 插件内选文不能传给网页助手。