MLLM4TS: Leveraging Vision and Multimodal Language Models for General Time-Series Analysis

TMLR Paper8198 Authors

Abstract: Effective analysis of time series data presents significant challenges due to the complex temporal dependencies and cross-channel interactions in multivariate data. Inspired by the way human analysts visually inspect time series to uncover hidden patterns, we ask: can incorporating visual representations enhance automated time-series analysis? Recent advances in multimodal large language models have demonstrated impressive generalization and visual understanding capability, yet their application to time series remains constrained by the modality gap between continuous numerical data and discrete natural language. To bridge this gap, we introduce MLLM4TS, a novel framework that leverages multimodal large language models for general time-series analysis by integrating a dedicated vision branch. Each time-series channel is rendered as a horizontally stacked color‑coded line plot in one composite image to capture spatial dependencies across channels, and a temporal‑aware visual patch alignment strategy then aligns visual patches with their corresponding time segments. MLLM4TS fuses fine-grained temporal details from the numerical data with global contextual information derived from the visual representation, providing a unified foundation for multimodal time-series analysis. Extensive experiments on standard benchmarks show that MLLM4TS consistently outperforms its unimodal counterpart across both predictive (e.g., classification) and generative (e.g., anomaly detection and forecasting) tasks, ranking among the top time-series backbones. These results highlight the effectiveness of introducing visual modalities and pretrained models for robust and generalizable time-series analysis.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Christopher_Mutschler1

Submission Number: 8198