Efficient distributed continual learning for steering experiments in real-time

Thomas Bouvier, Bogdan Nicolae, Alexandru Costan, Tekin Bicer, Ian T. Foster, Gabriel Antoniu

Abstract: Highlights•Motivate benefits of rehearsal-based continual learning for scientific applications.•Define rehearsal buffers and introduce extensions for data-parallel training.•Present key design principles, including asynchronous buffer management techniques.•Implement a distributed rehearsal buffer prototype integrated with PyTorch.•Report experiments on classification and generative tasks, showing improved accuracy.