Asynchronous Multi-Agent Reinforcement Learning for Collaborative Partial Charging in Wireless Rechargeable Sensor Networks

Yongheng Liang, Hejun Wu, Haitao Wang

Abstract: Online Scheduling for Partial charging with Multi-Mobile Chargers (OSPM) is critical for Wireless Rechargeable Sensor Networks (WRSNs) performing high-power monitoring tasks with a large number of simultaneous charging requests. However, existing studies for online scheduling assume full charging of sensors, leading to delays and inefficient resource utilization. Partially charging the sensors can improve scheduling efficiency and flexibility, but these studies focus on off-line scheduling, hindering dynamic decision-making. Multi-Agent Reinforcement Learning (MARL) is advantageous in online collaboration. Nevertheless, existing MARL methods assume synchronized actions, while Mobile Chargers (MCs) performing charging tasks asynchronously due to the difference in movement and charging times. On the other hand, hybrid actions are required to capture the simultaneous decision-making of MCs, involving sensor selection (discrete action) and energy allocation (continuous parameter). This introduces a circular dependency between a discrete action and its corresponding continuous parameter due to their interdependence. To deal with the above problems and address OSPM, we propose Asynchronous and Scalable Multi-agent Hybrid Proximal Policy Optimization (ASM-HPPO). The evaluation results not only indicate that our ASM-HPPO has advantages in terms of various performance metrics over existing schemes, but also demonstrate that our methods achieve higher stability and scalability.