Ticktack : Long Span Temporal Alignment of Large Language Models Leveraging Sexagenary Cycle Time Expression

ACL ARR 2025 May Submission2485 Authors

Abstract: Large language models (LLMs) suffer from temporal misalignment issues especially across long span of time. The issue stems from knowing that LLMs are trained on vast amounts of data with sparse temporal information over long periods, such as thousands of years, resulting in insufficient learning or catastrophic forgetting by the LLMs. This paper proposes a methodology named "Ticktack" for addressing the LLM's long-time span misalignment in a yearly setting. Specifically, we first propose to utilize the sexagenary year expression instead of the Gregorian year expression employed by LLMs, achieving a more uniform distribution in yearly granularity. Then, we employ polar coordinates to model the sexagenary cycle of 60 terms and the year order within each term, with additional temporal encoding to ensure LLMs understand them. Finally, we present a temporal representational alignment approach for post-training LLMs that effectively distinguishes time points with relevant knowledge, hence improving performance on time-related tasks, particularly over a long period. We also create a long time span benchmark for evaluation. Experimental results prove the effectiveness of our proposal.

Paper Type: Long

Research Area: Language Modeling

Research Area Keywords: Long-Span Temporal Alignment, Time expression, Large Language Model, Sexagenary year expression

Contribution Types: Approaches low compute settings-efficiency

Languages Studied: English, Chinese

Submission Number: 2485