## \title{Multi-Agent Reinforcement Learning Double \\ Auction Design for Online Advertising Recommender Systems}

Sep 29, 2021ICLR 2022 Conference Desk Rejected SubmissionReaders: Everyone
• Abstract: \begin{abstract} In recent years, real-time bidding (RTB) is a major way in the online display advertising recommender system. Diﬀerent mechanisms are good at addressing diﬀerent goals, real-time auctions are widely used for direct marketing. However, for advertisers how to optimize specific goals such as maximizing revenue and return on investment led by ad placements, not only need to estimate the relevance between the ads and user’s interests, but also require an efficient strategic response with respect to other advertisers bidding in the advertising recommender systems. Multi-agent reinforcement learning (MARL) studies how multiple agents interact in a common environment. In this paper, we formulate this joint bidding optimization problem using MARL, where each contract agent can submit bids for individual impressions. With the formulation, we optimal impression allocation strategy by multi-agent bidding functions for contracts. Since the bids from contracts are decided by the publisher, we propose a multi-agent reinforcement learning double auction (MARLDA) approach to balance the trade-off between the competition and cooperation among advertisers for the publisher to maximize its interest. The results demonstrate the effectiveness of our approach. \end{abstract}