XMC-Agent : Dynamic Navigation over Scalable Hierarchical Index for Incremental Extreme Multi-label Classification

Anonymous

Abstract: The eXtreme Multi-label Classification (XMC) aims at accurately assigning large-scale labels to instances, and is challenging for learning, managing, and predicting over the large-scale and rapidly growing set of labels. Traditional XMC methods, like one-vs-all and tree-based methods struggle with the growing set of labels due to their static label assumptions, and embedding-based methods struggle with the complex mapping relationships due to their late-interaction paradigm. In this paper, we propose a large language model (LLM) powered agent framework for extreme multi-label classification – \our{}, which can effectively learn, manage and predict the extremely large and dynamically increasing set of labels. Specifically, \our{} models the extreme multi-label classification task as a dynamic navigation problem, employing a scalable hierarchical label index to effectively manage the unified label space. Additionally, we propose two algorithms to enhance the dynamic navigation capabilities of \our{}: a self-construction algorithm for building the scalable hierarchical index, and an iterative feedback learning algorithm for adjusting the agent to specific tasks. Experiments show that \our{} achieves the state-of-the-art performance on three standard datasets.

Paper Type: long

Research Area: NLP Applications

Contribution Types: NLP engineering experiment

Languages Studied: english

Preprint Status: We plan to release a non-anonymous preprint in the next two months (i.e., during the reviewing process).

A1: yes

A1 Elaboration For Yes Or No: section 7

A2: no

A2 Elaboration For Yes Or No: Our work focuses on proposing a new solution paradigm for an existing task

A3: yes

A3 Elaboration For Yes Or No: section 1

B: no

B1: n/a

B2: no

B3: n/a

B4: n/a

B5: n/a

B6: yes

C: yes

C1: yes

C2: yes

C3: yes

C4: no

D: no

D1: no

D2: n/a

D3: n/a

D4: n/a

D5: n/a

E: yes

E1: n/a