Temporal Environment-Aware Image Generation via Latent Diffusion

Nasrin Kalanat, Yiqun Xie, Yanhua Li, Xiaowei Jia

Keywords: Generative models, Latent Diffusion, Time series data, Multi-modal learning

Abstract: Low-cost cameras have recently become widely used to monitor environmental ecosystems. This paper focuses on scene prediction for monitoring small streams, which is critical for ensuring water supply and informing early actions for floods and droughts. In contrast to traditional stream models that typically rely on coarse-resolution weather data, stream images provide detailed information about water properties and local environment at a higher temporal frequency. This paper presents a multi-modal generative framework designed for frequent temporal stream imagery datasets, aimed at generating the subsequent stream images. This task is challenging due to the variability of stream images caused by changes in time and local environmental conditions. Our method captures scene changes in both stream and surrounding environment by incorporating temporal context of weather, water flow, and time information. We also introduce a domain-discriminative learning approach to enforce the learning of domain-specific information in generating images. Our experiments demonstrate the superior performance of the proposed method in preserving semantics of water and environmental properties, using real data from the West Brook area in western Massachusetts, USA.

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 11534