arxiv.org·2026/10/26
为什么重要 为化学文献反应提取提供细粒度基准与解耦协议，方便相关多模态模型评测与数据构建。
Screen Tables 和 Substrate Scopes 是合成化学文献中广泛使用的信息密集图，但从它们中自动提取结构化反应数据仍具挑战性。我们提出 RxnExBench，一个用于细粒度反应级评测的多模态基准。
该基准基于高影响力化学期刊构建，包含 3,565 个专家标注的反应和 2,361 个分子结构。数据集采用包含 15 个反应角色和 26 个核心字段的统一模式，支持下游 AI-for-Chemistry 任务，如逆合成规划、产率预测和反应条件推荐。
RxnExBench 还提供推理驱动的难度分类、模块化标注流程和 Oracle-MolRec 协议，该协议将分子识别与更高层推理解耦。对 9 个最先进 VLM 的评测显示，随着模式复杂性和推理需求增加，性能大幅下降。在 Oracle-MolRec 设置下，结果进一步表明仅靠准确的分子识别是不够的，多模态整合和化学感知推理仍是主要瓶颈。我们发布所有数据和代码，以促进化学感知多模态理解和 AI-for-Chemistry 数据集构建的未来研究。数据集和代码将在 https://github.com/opendatalab/RxnExBench 发布。
Screen Tables and Substrate Scopes are information-dense figures widely used in synthetic chemistry literature, yet automated extraction of structured reaction data from them remains challenging.
We introduce RxnExBench, a multimodal benchmark for fine-grained reaction-level evaluation.
Built from high-impact chemistry journals, it contains 3,565 expert-annotated reactions and 2,361 molecular structures.
The dataset adopts a unified schema with 15 reaction roles and 26 core fields, supporting downstream AI-for-Chemistry tasks such as retrosynthetic planning, yield prediction, and reaction condition recommendation.
RxnExBench further provides a reasoning-driven difficulty taxonomy, a modular annotation pipeline, and the Oracle-MolRec protocol, which disentangles molecular recognition from higher-level reasoning.
Evaluation of 9 state-of-the-art VLMs shows that performance degrades substantially as schema complexity and reasoning demands increase.
Under the Oracle-MolRec setting, results further reveal that accurate molecular recognition alone is insufficient, with multimodal integration and chemistry-aware reasoning remaining the primary bottlenecks.
We release all data and code to facilitate future research in chemistry-aware multimodal understanding and AI-for-Chemistry dataset construction.
The dataset and code will be released at https://github.com/opendatalab/RxnExBench.
原版排版与图表 · 手机无法内嵌时可在浏览器打开。PDF 插件内选文不能传给网页助手。