MONITRS: Multimodal Observations of Natural Incidents Through Remote Sensing

Shreelekha Revankar, Utkarsh Mall, Cheng Perng Phoo, Kavita Bala, Bharath Hariharan

Keywords: Satellite Imagery Natural Disasters

TL;DR: Multimodal satellite imagery dataset for natural disaster monitoring

Abstract: Natural disasters cause devastating damage to communities and infrastructure every year. Effective disaster response is hampered by the difficulty of accessing affected areas during and after events. Remote sensing has allowed us to monitor natural disasters in a remote way. More recently there have been advances in computer vision and deep learning that help automate satellite imagery analysis, However, they remain limited by their narrow focus on specific disaster types, reliance on manual expert interpretation, and lack of datasets with sufficient temporal granularity or natural language annotations for tracking disaster progression. We present MONITRS, a novel multimodal dataset of $\sim$10,000 FEMA disaster events with temporal satellite imagery with natural language annotations from news articles, accompanied by geotagged locations, and question-answer pairs. We demonstrate that fine-tuning existing MLLMs on our dataset yields significant performance improvements for disaster monitoring tasks, establishing a new benchmark for machine learning-assisted disaster response systems.

Dataset URL: https://huggingface.co/datasets/ShreelekhaR/MONITRS

Code URL: https://github.com/ShreelekhaR/MONITRS

Primary Area: Datasets & Benchmarks for applications in language modeling and vision language modeling

Submission Number: 1402