Multi-Agent Reinforcement Learning: Independent versus Cooperative Agents

Ming Tan

Abstract: Intelligent human agents exist in a cooperative social environment that facilitates learning. They learn not only by trial-and-error, but also through cooperation by sharing instantaneous information, episodic experience, and learned knowledge. The key investigations of this paper are, “Given the same number of reinforcement learning agents, will cooperative agents outperform independent agents who do not communicate during learning?” and “What is the price for such cooperation?” Using independent agents as a benchmark, cooperative agents are studied in following ways: (1) sharing sensation, (2) sharing episodes, and (3) sharing learned policies. This paper shows that (a) additional sensation from another agent is beneficial if it can be used efficiently, (b) sharing learned policies or episodes among agents speeds up learning at the cost of communication, and (c) for joint tasks, agents engaging in partnership can significantly outperform independent agents although they may learn slowly in the beginning. These tradeoff's are not just limited to multi-agent reinforcement learning. Previous chapter in book Next chapter in book Recommended articles Citing articles (0) Copyright © 1993 Morgan Kaufmann Publishers, Inc. Published by Elsevier Inc. All rights reserved. Recommended articles High degrees in recursive trees Electronic Notes in Discrete Mathematics, Volume 50, 2015, pp. 451-456 Download PDF View details Reinforcement learning approaches for specifying ordering policies of perishable inventory systems Expert Systems with Applications, Volume 91, 2018, pp. 150-158 Download PDF View details Network coding based joint signaling and dynamic bandwidth allocation scheme for inter optical network unit communication in passive optical networks Optical Fiber Technology, Volume 20, Issue 3, 2014, pp. 280-293 Download PDF View details 1 2 Next Citing articles (0) Article Metrics Citations Citation Indexes: 263 Captures Readers: 513 View details Elsevier About ScienceDirect Remote access Shopping cart Advertise Contact and support Terms and conditions Privacy policy We use cookies to help provide and enhance our service and tailor content and ads. By continuing you agree to the use of cookies . Copyright © 2020 Elsevier B.V. or its licensors or contributors. ScienceDirect ® is a registered trademark of Elsevier B.V. ScienceDirect ® is a registered trademark of Elsevier B.V. {"abstracts":{"content":[{"$$":[{"$":{"id":"cesectitle1"},"#name":"section-title","_":"Abstract"},{"$$":[{"$$":[{"#name":"__text__","_":"Intelligent human agents exist in a cooperative social environment that facilitates learning. They learn not only by trial-and-error, but also through "},{"#name":"italic","_":"cooperation"},{"#name":"__text__","_":" by sharing instantaneous information, episodic experience, and learned knowledge. The key investigations of this paper are, “Given the same number of reinforcement learning agents, will cooperative agents outperform independent agents who do not communicate during learning?” and “What is the price for such cooperation?” Using independent agents as a benchmark, cooperative agents are studied in following ways: (1) sharing sensation, (2) sharing episodes, and (3) sharing learned policies. This paper shows that (a) additional sensation from another agent is beneficial if it can be used efficiently, (b) sharing learned policies or episodes among agents speeds up learning at the cost of communication, and (c) for joint tasks, agents engaging in partnership can significantly outperform independent agents although they may learn slowly in the beginning. These tradeoff's are not just limited to multi-agent reinforcement learning."}],"$":{"view":"all","id":"spara1"},"#name":"simple-para"}],"$":{"view":"all","id":"abssec1"},"#name":"abstract-sec"}],"$":{"xmlns:ce":true,"view":"all","id":"abs1","class":"author"},"#name":"abstract"}],"floats":[],"footnotes":[],"attachments":[]},"accessOptions":{},"adobeTarget":{"variation":"control"},"article":{"publication-content":{"publicationCity":"San Francisco (CA)","publisherName":"Morgan Kaufmann","coverImageUrl":"https://ars.els-cdn.com/content/image/Dmorgank.gif","transactionsBlocked":"false","coverThumbnailUrl":"3-s2.0-B9781558603073-cov150h.gif","specialCoverImage":"3-s2.0-C20090277981-cov150h.gif","sourceOpenAccess":false,"publicationCoverImageUrl":"https://ars.els-cdn.com/content/image/3-s2.0-C20090277981-cov150h.gif"},"pii":"B9781558603073500496","dates":{"Available online":"27 June 2014","Revised":[],"Publication date":"1 January 1993"},"access":{"openArchive":false,"openAccess":false},"crawlerInformation":{"canCrawlPDFContent":false,"isCrawler":false},"document-references":15,"accessOptions":{"accessHeader":{"parameters":[],"key":"access_header_no_remote_access"},"outwardLinksSection":{"linkingHubUrl":"https://linkinghub.elsevier.com/retrieve/pii/B9781558603073500496?showall