LPMARL: Linear Programming-based Task Assignment for Hierarchical Multi-agent Reinforcement Learning

Kyuree Ahn, Jinkyoo Park

Keywords: Multi-agent reinforcement learning, Implicit deep learning, Hierarchical multi-agent reinforcement learning

TL;DR: LPMARL is a hierarchical MARL method that leverages linear programming for agent–task assignment and optimizes both high- and low-level policies end-to-end to tackle cooperative games with sparse rewards.

Abstract: Training Multi-Agent Reinforcement Learning (MARL) with sparse rewards is challenging due to complex agent interactions. We propose Linear Programming-based hierarchical MARL (LPMARL), which integrates constrained optimization into MARL for effective coordination. LPMARL operates in two stages: (1) solving agent–task assignment via a Linear Program with state-dependent costs from a Graph Neural Network (GNN), and (2) solving cooperative sub-games among assigned agents. Both the LP generator and low-level policies are trained end-to-end by differentiating through the optimization layer. Experiments show that LPMARL achieves effective task allocation and sub-policy learning across diverse cooperative games.

Submission Number: 43