Kairos: Redefining Event and Time Prediction with Language Modeling

Akash Gupta, Sahil Verma, Sumit Bisht, Nitika Verma, Purav Aggarwal, Venkata Phanindra Kumar Grandhi, Abhishek Persad

Abstract: Continuous time-event sequence (CTES) forecasting is essential across diverse domains, from healthcare to finance, requiring accurate prediction of both future event types and their timestamps. Traditionally, CTES forecasting has been driven by Temporal Point Processes (TPPs), which rely on intensity function-based priors. However, these methods often fail to generalize effectively to real-world scenarios as well as perform poorly over longer horizons. While recent diffusion-based approaches are promising, they are limited by the need to define a fixed prior while training, such as number of events to forecast or time horizon, which requires training multiple models for different horizon lengths. We present Kairos, a novel model that reformulates CTES forecasting as a language modeling task. Our model employs a decoder-only transformer architecture with a unified tokenization approach that represents time and events in a shared embedding space. By structuring the input as alternating event and time tokens, the model learns to capture the inherent temporal relationships between events. Through comprehensive experiments on multiple large-scale datasets, we demonstrate that Kairos consistently outperforms state-of-the-art baselines, achieving average improvements of 4.5% and 7.8% in short term forecasting for event and time respectively and 14.41% improvement in long term forecasting. Additionally, we conduct extensive ablation studies and qualitative analysis to understand the inner workings of Kairos.

Submission Number: 124