Unraveling Complexity: An Exploration Into Large-Scale Multimodal Signal Processing

Zhenyu Wen, Yuheng Ye, Jie Su, Taotao Li, Jinhao Wan, Shilian Zheng, Zhen Hong, Shibo He, Haoran Duan, Yuexiang Li, Yawen Huang, Yefeng Zheng

Abstract: Advanced communication systems and military reconnaissance are increasingly prevalent in high-tech environments, greatly supported by the flourishing in signal processing technologies. The recent exponential proliferation of sensors led to an unprecedented expansion in the scale and diversity of signals across various modalities. Such an influx poses significant challenges in effectively integrating multimodal signal data to deliver comprehensive and interpretive solutions across a diverse range of applications. In this article, we provide an overview of the core issues, challenges, and future research directions in different stages of developing large-scale multimodal signal processing models. Additionally, we introduce a prior investigation into signal representation learning, where we propose a contrastive-learning-based framework to extract fine-grained signal features under few-shot conditions. Our proposed framework achieves a 24.1% performance improvement over baseline approaches, consistently demonstrating superiority over state-of-the-art methods. The code is accessible in this repository: https://github.com/YYH211/LSM.