SOP2: Transfer Learning with Scene-Oriented Prompt Pool on 3D Object Detection

Ching-Hung Cheng, Hsiu-Fu Wu, Bing-Chen Wu, Khanh-Phong Bui, Van-Tin Luu, Ching-Chun Huang

Abstract:

With the rise of Large Language Models (LLMs) such
as GPT-3, these models exhibit strong generalization ca-
pabilities. Through transfer learning techniques such as
fine-tuning and prompt tuning, they can be adapted to vari-
ous downstream tasks with minimal parameter adjustments.
This approach is particularly common in the field of Natu-
ral Language Processing (NLP). This paper aims to explore
the effectiveness of common prompt tuning methods in 3D
object detection. We investigate whether a model trained
on the large-scale Waymo dataset can serve as a foundation
model and adapt to other scenarios within the 3D object de-
tection field. This paper sequentially examines the impact of
prompt tokens and prompt generators, and further proposes
a Scene-Oriented Prompt Pool (SOP2). We demonstrate the
effectiveness of prompt pools in 3D object detection, with
the goal of inspiring future researchers to delve deeper into
the potential of prompts in the 3D field.