Bridging Theory and Practice in Multimodal Deep Learning: A Comprehensive Review in the Large Language Model Era

Anonymous

Abstract: In the past few years, the realm of deep learning has captivated widespread interest, with multimodal deep learning (MMDL) rising as an exceptionally promising area. MMDL specializes in processing and amalgamating data from varied communication channels, including text, speech, vision, and spatial indicators. This article delivers an exhaustive exploration of MMDL methodologies and their expansive applications. Furthermore, we delve into a detailed examination of diverse MMDL techniques, encapsulating the progression of model architectures, advancements in data augmentation, refresh methods, and optimization tactics. The main goal of this review is to tackle the pressing challenges and delineate the trajectory for future research in the dynamic field of deep learning, especially focusing on the era of Large Language Models (LLMs). We believe that this comprehensive review will greatly enhance the comprehension of MMDL and act as a crucial tool for researchers aiming to delve into new and promising research paths.

Paper Type: long

Research Area: Multimodality and Language Grounding to Vision, Robotics and Beyond

Contribution Types: Surveys

Languages Studied: English