Explaining and Tuning Transformer-based LLMs in Arithmetic Tasks with Human Strategies

ACL ARR 2025 July Submission272 Authors

Abstract: Transformer-based large language models continue to achieve SOTA performance across various natural language processing tasks. However, their subpar performance on seemingly elementary problems, such as basic arithmetic, raises concerns about model reliability, safety, and ethical deployment. In this study, we propose a comprehensive problem analysis framework to explain the underlying mechanisms of vanilla Transformer model trained on integer arithmetic tasks, and tune the model using strategies effective for humans. We begin by decomposing the arithmetic into well-defined subtasks commonly used by humans and conducting loss convergence order analysis together with ablation studies for each subtask. Our findings reveal that LLMs exhibit learning patterns similar to those of humans, with a faster learning speed for simpler subtasks compared to complex ones. In addition, we successfully improved the accuracy of LLMs by applying human arithmetic strategies. These results suggest that transformers may share similar information processing mechanisms to humans in arithmetic. Our work has important implications for enhancing LLMs' arithmetic abilities by applying human strategies and understanding LLMs using explainable AI verifications and comparisons with humans, ultimately fostering trust in LLMs for critical and high-stakes applications.

Paper Type: Long

Research Area: Interpretability and Analysis of Models for NLP

Research Area Keywords: Explainable AI, large language model, transformer, Interpretability, Arithmetic

Contribution Types: Model analysis & interpretability

Languages Studied: English, math

Reassignment Request Area Chair: This is not a resubmission

Reassignment Request Reviewers: This is not a resubmission

A1 Limitations Section: This paper has a limitations section.

A2 Potential Risks: N/A

B Use Or Create Scientific Artifacts: Yes

B1 Cite Creators Of Artifacts: N/A

B2 Discuss The License For Artifacts: N/A

B3 Artifact Use Consistent With Intended Use: N/A

B4 Data Contains Personally Identifying Info Or Offensive Content: N/A

B5 Documentation Of Artifacts: N/A

B6 Statistics For Data: N/A

C Computational Experiments: Yes

C1 Model Size And Budget: Yes

C1 Elaboration: We provide the details of model hyperparameter and data used in our experiments in section3 and implementation details section. And the number of parameters and total computational budget can be computed by the information provided in our paper.

C2 Experimental Setup And Hyperparameters: Yes

C2 Elaboration: We provide the implementation details in Appendix-B, which consists of the model details.

C3 Descriptive Statistics: Yes

C3 Elaboration: All our experiments in section 4 and Appendix are repeated several times and report the average result.

C4 Parameters For Packages: Yes

C4 Elaboration: We provide the implementation details in Appendix-B

D Human Subjects Including Annotators: No

D1 Instructions Given To Participants: N/A

D2 Recruitment And Payment: N/A

D3 Data Consent: N/A

D4 Ethics Review Board Approval: N/A

D5 Characteristics Of Annotators: N/A

E Ai Assistants In Research Or Writing: Yes

E1 Information About Use Of Ai Assistants: Yes

E1 Elaboration: In section 1, we mention the usage of ChatGPT

Author Submission Checklist: yes

Submission Number: 272