Efficient Parallelized Simulation of Cyber-Physical Systems

Published: 09 May 2024, Last Modified: 09 May 2024Accepted by TMLREveryoneRevisionsBibTeX
Abstract: Advancements in accelerated physics simulations have greatly reduced training times for reinforcement learning policies, yet the conventional step-by-step agent-simulator interaction undermines simulation accuracy. In the real-world, interactions are asynchronous, with sensing, acting and processing happening simultaneously. Failing to capture this widens the sim2real gap and results in suboptimal real-world performance. In this paper, we address the challenges of simulating realistic asynchronicity and delays within parallelized simulations, crucial to bridging the sim2real gap in complex cyber-physical systems. Our approach efficiently parallelizes cyber-physical system simulations on accelerator hardware, including physics, sensors, actuators, processing components and their asynchronous interactions. We extend existing accelerated physics simulations with latency simulation capabilities by constructing a `supergraph' that encodes all data dependencies across parallelized simulation steps, ensuring accurate simulation. By finding the smallest supergraph, we minimize redundant computation. We validate our approach on two real-world systems and perform an extensive ablation, demonstrating superior performance compared to baseline methods.
Certifications: Reproducibility Certification
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: Changes are highlighted in blue, with the main ones listed below: - Correct typographical error in the effective sampling time. - Add missing edges in Fig. 3. - Expanded the scalability analysis to include V2V Platooning and UAV swarm control topologies. - Improved manuscript clarity for an ML audience and updated figures for better intuitiveness. - Motivated the speed evaluation approach that excludes overhead related to the learning algorithm. - Expanded experimental validation with sim-to-delay-sim experiments. - Clarified the acyclicity of Algorithm 2 in Section 3.2 and 3.3. - Clarified the aim of sim2real experiments to demonstrate the capability of delay simulation through predication mask randomization. - Indicated points of approximation within the algorithm in Algorithm 1. - Discussed unexpected performance findings between the "linear" and "power" approaches in Appendix B. - Moved the Scalability and Ablation study with abstract topologies to Appendices A and B, respectively. - Visualized the computation graph and supergraphs in Appendix C. - Added a version of the compiler in supplementary material to enhance accessibility for ML community. - Add a clear definition of CPS and a description of one application to the introduction.
Video: https://youtu.be/I-1asHKBX6o
Code: https://github.com/bheijden/supergraph
Supplementary Material: zip
Assigned Action Editor: ~Adam_M_White1
Submission Number: 2097