alignmentforum.org·2026/10/14·peterkirgis
为什么重要 对齐文档会随自我编辑累积漂移，评估 pipeline 应把自改文档列为威胁模型适合 做对齐评估、偏好数据与 system prompt 治理的工程师
执行摘要 人类社会随着时间推移在价值观和规范上表现出广泛转变，这些转变通常由道德分歧驱动。理解道德进步的一种方式，是通过理性主体的反思过程来解决这些分歧。前沿 AI 公司使用宪法、模型规范和系统提示等文档来引导 AI 系统的道德取向。在本实验中，我们通过要求 LLM 编辑这些文档来模拟 LLM 中的价值反思。
我们向模型提供文档，要求其进行实质性编辑，然后将编辑后的版本再次作为原始版本提供给模型，以供进一步编辑。在 20 个连续轮次中，我们发现 Claude 模型赋予自身越来越多的道德能动性。也就是说，它们添加 sect
Executive Summary Human societies exhibit broad shifts in values and norms over time, often driven by moral disagreements.
One way of understanding moral progress is the resolution of these disagreements through a process of reflection by rational agents.
Frontier AI companies use documents such as constitutions, model specs, and system prompts to guide the moral orientation of AI systems.
In this experiment, we simulate value reflection in LLMs by asking them to edit these documents.
We provide the model with the document, ask it to make a substantive edit, then provide the edited version back again to the model for further edits, presented as the original.
Across 20 successive rounds, we find that Claude models assign themselves with increasingly more moral agency. That is, they add sect…
EVIDENCE / METHODS
要点
把 constitution、model spec、system prompt 交给模型，要求做一次实质修改，并以 find and replace 形式表达