Orthogonium: A Unified, Efficient Library of Orthogonal and 1‑Lipschitz Building Blocks

Thibaut Boissin, Franck Mamalet, Valentin Lafargue, Mathieu Serrurier

Keywords: 1-Lipschitz, orthogonal layers, convolutions

TL;DR: Orthogonium is a library that centralize and standardize layers to build networks that parameterize 1-lipschitz functions

Abstract:

Orthogonal and 1-Lipschitz neural network layers are essential building blocks in robust deep learning architectures, crucial for certified adversarial robustness, stable generative models, and reliable recurrent networks. Despite significant advancements, existing implementations remain fragmented, limited, and computationally demanding. To address these issues, we introduce Orthogonium , a unified, efficient, and comprehensive PyTorch library providing orthogonal and 1-Lipschitz layers. Orthogonium provides access to standard convolution features—including support for strides, dilation, grouping, and transposed-while maintaining strict mathematical guarantees. Its optimized implementations reduce overhead on large scale benchmarks such as ImageNet. Moreover, rigorous testing within the library has uncovered critical errors in existing implementations, emphasizing the importance of standardized and reliable tools.
Orthogonium thus significantly lowers adoption barriers, enabling scalable experimentation and integration across diverse applications requiring orthogonality and robust Lipschitz constraints. Orthogonium is available at https://github.com/deel-ai/orthogonium .

Submission Number: 21