摘要
思维链（CoT）蒸馏通常仅通过最终答案准确率评估小模型，忽视了推理轨迹质量。本文在医疗问答场景中发现，经蒸馏的学生模型虽显著提升了答案准确率并降低了校准误差，但其推理步骤的错误率却大幅上升。这种“答案质量提升但轨迹事实性下降”的现象在不同评估者、模型规模及基准测试中普遍存在。研究表明，当紧凑的答案无法充分约束理由时，学生模型可能模仿专家形式却缺乏局部主张的可靠依据，仅靠答案级指标无法揭示此风险。
AI 推荐理由
论文核心研究思维链蒸馏中推理轨迹的质量与准确性背离现象，深入评估推理过程。
研究机构
School of Health & Wellbeing, University of Glasgow, Glasgow, UK
Department of Respiratory and Critical Care Medicine, Shanghai Sixth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
Institute of Health Informatics, University College London, London, UK
论文信息