Efficient Parallelized Simulation of Cyber-Physical Systems

Bas van der Heijden, Laura Ferranti, Jens Kober, Robert Babuska

Abstract: Advancements in accelerated physics simulations have greatly reduced training times for reinforcement learning policies, yet the conventional step-by-step agent-simulator interaction undermines simulation accuracy. In the real-world, interactions are asynchronous, with sensing, acting and processing happening simultaneously. Failing to capture this widens the sim2real gap and results in suboptimal real-world performance. In this paper, we address the challenges of simulating realistic asynchronicity and delays within parallelized simulations, crucial to bridging the sim2real gap in complex cyber-physical systems. Our approach efficiently parallelizes cyber-physical system simulations on accelerator hardware, including physics, sensors, actuators, processing components and their asynchronous interactions. We extend existing accelerated physics simulations with latency simulation capabilities by constructing a `supergraph' that encodes all data dependencies across parallelized simulation steps, ensuring accurate simulation. By finding the smallest supergraph, we minimize redundant computation. We validate our approach on two real-world systems and perform an extensive ablation, demonstrating superior performance compared to baseline methods.

Certifications: Reproducibility Certification

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

Changes are highlighted in blue, with the main ones listed below:

• Correct typographical error in the effective sampling time.
• Add missing edges in Fig. 3.
• Expanded the scalability analysis to include V2V Platooning and UAV swarm control topologies.
• Improved manuscript clarity for an ML audience and updated figures for better intuitiveness.
• Motivated the speed evaluation approach that excludes overhead related to the learning algorithm.
• Expanded experimental validation with sim-to-delay-sim experiments.
• Clarified the acyclicity of Algorithm 2 in Section 3.2 and 3.3.
• Clarified the aim of sim2real experiments to demonstrate the capability of delay simulation through predication mask randomization.
• Indicated points of approximation within the algorithm in Algorithm 1.
• Discussed unexpected performance findings between the "linear" and "power" approaches in Appendix B.
• Moved the Scalability and Ablation study with abstract topologies to Appendices A and B, respectively.
• Visualized the computation graph and supergraphs in Appendix C.
• Added a version of the compiler in supplementary material to enhance accessibility for ML community.
• Add a clear definition of CPS and a description of one application to the introduction.

Video: https://youtu.be/I-1asHKBX6o

Code: https://github.com/bheijden/supergraph

Assigned Action Editor: Adam M White

Submission Number: 2097