Enhancing Research Productivity: Seamless Integration of Personal Devices and HPC Resources with the Cybershuttle Notebook Gateway

Yasith Jayawardana, Dimuthu Wannipurage, Eroma Abeysinghe, Suresh Marru

Abstract: Scientists often utilize personal laptops and workstations for initial research stages and turn to high-performance computing (HPC) supercomputers for compute-intensive tasks. However, seamless transitions between these environments are vital for enhancing productivity and accelerating research progress. Our paper presents the Cybershuttle Notebook Gateway, an open-source framework crafted to streamline this transition, optimize resource utilization, and reduce time-to-science for researchers. Leveraging JupyterLab, the framework extends kernel mechanics for seamless provisioning and connection to remote HPC cluster kernels. We delve into its architecture, which separates user authentication, kernel provisioning, and remote file system access. Additionally, we highlight practical capabilities like analyzing network load and CPU usage across various computational workloads.