CLLMate: A Multimodal Benchmark for Weather and Climate Events Forecasting

ACL ARR 2025 February Submission2959 Authors

Abstract: Forecasting weather and climate events is crucial for making appropriate measures to mitigate environmental hazards and minimize losses. However, existing environmental forecasting research focuses narrowly on predicting numerical meteorological variables (e.g., temperature), neglecting the translation of these variables into actionable textual narratives of events and their consequences. To bridge this gap, we proposed Weather and Climate Event Forecasting (WCEF), a new task that leverages numerical meteorological raster data and textual event data to predict weather and climate events. This task is challenging to accomplish due to difficulties in aligning multimodal data and the lack of supervised datasets. To address these challenges, we present CLLMate, the first multimodal dataset for WCEF, using 26,156 environmental news articles aligned with ERA5 reanalysis data. We systematically benchmark 23 existing MLLMs on CLLMate, including closed-source, open-source, and our fine-tuned models. Our experiments reveal the advantages and limitations of existing MLLMs and the value of CLLMate for the training and benchmarking of the WCEF task.

Paper Type: Long

Research Area: Resources and Evaluation

Research Area Keywords: LLM-based forecasting, Multimodal LLM, Multimodal Alignment, Weather and climate event forecasting

Contribution Types: Data resources

Languages Studied: English

Submission Number: 2959