Response-Time Analysis of a Soft Real-time NVIDIA Holoscan Application

Philip Schowitz, Soham Sinha, Arpan Gujarati

Abstract: NVIDIA Holoscan SDK is a novel edge and embedded software development framework designed for NVIDIA System-on-Chips (SoCs), primarily targeting medical device applications. This SDK facilitates complex data processing workflows using Directed Acyclic Graphs (DAGs) composed of functional units termed operators. These operators, running in separate threads, are usually interconnected with intricate execution dependencies influenced by both upstream and downstream conditions on communication data buffers. Current methods to measure the response time of a complex Holoscan application rely on empirical benchmarking, which can be costly, time-consuming, and unreliable – limitations that are particularly critical in sectors where safety and certification concerns are paramount. This paper introduces a novel static analysis methodology to determine worst-case end-to-end response times in NVIDIA Holoscan applications. Our approach overcomes the drawbacks of existing empirical tools by providing a response-time analysis capable of handling complex operator interactions and communication buffering mechanisms inherent in Holoscan’s architecture. Through rigorous theoretical analysis and empirical validation, our method not only ensures predictability in system behavior but also aids developers in identifying performance bottlenecks and optimizing system design. Evaluation using real-world NVIDIA HoloHub applications demonstrates the efficiency and accuracy of our analysis, achieving theoretical response times as close as $0.3 \%$ of empirically measured numbers on NVIDIA hardware using less than 1ms computation time.