S2AF: An action framework to self-check the Understanding Self-Consistency of Large Language Models

Huihui Shao, Fanyu Wang, Zhenping Xie

Abstract: Highlights•A novel perspective to evaluate the mastery of knowledge relationships in LLMs.•Propose S2AF<math><mrow is="true"><msup is="true"><mrow is="true"><mi is="true">S</mi></mrow><mrow is="true"><mn is="true">2</mn></mrow></msup><mi is="true">A</mi><mi is="true">F</mi></mrow></math>, a four-stage self-check framework for evaluating LLM self-consistency.•Benchmark LLM reasoning via experiments on FOLIO and LogicAware across three paradigms.•Show SEF improves reasoning by leveraging LLM outputs for self-enhanced feedback.•Evaluate six LLMs, revealing performance gaps and insights for better knowledge consistency.