Efficient Parallelized Simulation of Cyber-Physical Systems

Bas van der Heijden, Laura Ferranti, Jens Kober, Robert Babuska

Abstract: Advancements in accelerated physics simulations have greatly reduced training times for reinforcement learning policies, yet the conventional step-by-step agent-simulator interaction undermines simulation accuracy. In the real-world, interactions are asynchronous, with sensing, acting and processing happening simultaneously. Failing to capture this widens the sim2real gap and results in suboptimal real-world performance. In this paper, we address the challenges of simulating realistic asynchronicity and delays within parallelized simulations, crucial to bridging the sim2real gap in complex cyber-physical systems. Our approach efficiently parallelizes cyber-physical system simulations on accelerator hardware, including physics, sensors, actuators, processing components and their asynchronous interactions. We extend existing accelerated physics simulations with latency simulation capabilities by constructing a `supergraph' that encodes all data dependencies across parallelized simulation steps, ensuring accurate simulation. By finding the smallest supergraph, we minimize redundant computation. We validate our approach on two real-world systems and perform an extensive ablation, demonstrating superior performance compared to baseline methods.

Certifications: Reproducibility Certification

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

Changes are highlighted in blue, with the main ones listed below:
- Correct typographical error in the effective sampling time.
- Add missing edges in Fig. 3.
- Expanded the scalability analysis to include V2V Platooning and UAV swarm control topologies.
- Improved manuscript clarity for an ML audience and updated figures for better intuitiveness.
- Motivated the speed evaluation approach that excludes overhead related to the learning algorithm.
- Expanded experimental validation with sim-to-delay-sim experiments.
- Clarified the acyclicity of Algorithm 2 in Section 3.2 and 3.3.
- Clarified the aim of sim2real experiments to demonstrate the capability of delay simulation through predication mask randomization.
- Indicated points of approximation within the algorithm in Algorithm 1.
- Discussed unexpected performance findings between the "linear" and "power" approaches in Appendix B.
- Moved the Scalability and Ablation study with abstract topologies to Appendices A and B, respectively.
- Visualized the computation graph and supergraphs in Appendix C.
- Added a version of the compiler in supplementary material to enhance accessibility for ML community.
- Add a clear definition of CPS and a description of one application to the introduction.

Video: https://youtu.be/I-1asHKBX6o

Code: https://github.com/bheijden/supergraph

Assigned Action Editor: ~Adam_M_White1

Submission Number: 2097