arxiv.org·2026/10/26
现有 MLLM 驾驶模型每帧从零生成单一解释，场景模糊时过早锁定错误假设并在闭环中强化误判。DriveMind 改为维护多个自然语言假设的时变信念状态，以粒子滤波式传播、证据一致性重加权和重采样更新；慢速 VLM 推理器负责多假设推理与机动分布，快速扩散控制器按加权信念嵌入生成短时航点轨迹，二者异步运行。训练分机动标签与人工/Claude 一致性评分两阶段监督。在 Bench2Drive 闭环评测中，相较同管线去掉信念跟踪的基线，作者报告 DS 由 68.50 升至 81.37，SR 由 41.36% 升至 54.09%，收益集中在无信号交叉口、对向交通等需意图消歧场景，NAVSIM 的 PDMS 同样提升。消融表明维持多个假设、VLM 评分加权和异步刷新均有效，共享 GPU 下异步控制延迟低于同步基线；局限是假设数与更新频率固定，且尚未显式建模其他智能体的未来意图。
为什么重要 以多假设持续更新替代逐帧从零推理，在模糊交互场景提升显著，对闭环驾驶决策设计有直接借鉴价值。
近期多模态大语言模型（MLLMs）能在单一模型内感知交通场景、对其进行推理并提出驾驶动作。然而，现有基于 MLLM 的驾驶者在每一帧从头开始推理，并只给出单一解释，丢弃了先前关于场景的假设。当场景模糊时，智能体锁定在错误的解释上，并在证据积累之前据此行动。
本文提出 DriveMind，一个基于 MLLM 的驾驶框架，同时保持对场景的多个假设，并在新帧到来时更新它们的权重。受粒子滤波启发，DriveMind 通过 VLM 传播每个假设，根据当前观测对其打分，并重采样以保持多样性。
在 Bench2Drive 上，将信念跟踪加入同一驾驶管道，将 Driving Score（DS）从 68.50 提升到 81.37，将 Success Rate（SR）从 41.36% 提升到 54.09%。最大提升出现在模糊场景，例如无信号交叉口（+20.0 DS）和对向来车交互（+20.5 DS）。除总体指标外，DriveMind 还改进了细粒度交互驾驶能力，如并道、紧急制动和让行。
Recent multimodal large language models (MLLMs) can perceive a traffic scene, reason about it, and propose a driving action within a single model.
However, existing MLLM-based drivers reason from scratch at each frame and commit to a single explanation, discarding earlier hypotheses about the scene.
When the scene is ambiguous, the agent locks in on the wrong interpretation and acts on it before evidence accumulates.
In this paper, we propose DriveMind, an MLLM-based driving framework that keeps several hypotheses about the scene alive at the same time and updates their weights as new frames arrive.
Inspired by particle filtering, DriveMind propagates each hypothesis through a VLM, scores it against the current observation, and resamples to maintain diversity.
On Bench2Drive, adding belief tracking to the same driving pipeline improves the Driving Score (DS) from 68.50 to 81.37 and the Success Rate (SR) from 41.36% to 54.09%.
The largest gains occur on ambiguous scenarios such as unsignalized intersections (+20.0 DS) and oncoming-traffic interactions (+20.5 DS).
Beyond the overall metrics, DriveMind also improves fine-grained interactive driving abilities such as merging, emergency braking, and give-way.
原版排版与图表 · 手机无法内嵌时可在浏览器打开。PDF 插件内选文不能传给网页助手。