DISK: Differentiable Sparse Kernel Complex for Efficient Spatially-Variant Convolution

Zhizhen Wu, Cao Zhe, Yuchi Huo

Keywords: Kernel Approximation, Differentiable Filtering, Spatially-Varying Convolution, Efficient Image Processing

TL;DR: This work introduces a differentiable framework for decomposing complex kernels into optimized sparse layers, enabling high-performance, spatially-varying filtering via a filter-space interpolation scheme.

Abstract:

Image convolution with complex kernels is common in photography, scientific imaging, and animation, but dense convolution is too expensive for resource-limited devices. Existing approximations, such as simulated annealing and low-rank decompositions, are either slow or struggle with non-convex kernels.
We present a differentiable kernel decomposition framework that represents a spatially variant dense kernel with a small set of sparse samples, assuming the target dense kernel is known for both optimization and filtering. Our method provides (i) end-to-end differentiable sparse-kernel optimization, (ii) shape-aware initialization for non-convex kernels, and (iii) kernel-space interpolation for efficient, multi-dimensional spatially varying filtering without retraining or added runtime cost.
Across Gaussian and non-convex kernels, our method achieves higher fidelity than simulated annealing and lower cost than low-rank decomposition. It is practical for mobile imaging and real-time rendering, and integrates cleanly into learning pipelines.

Primary Area: optimization

Submission Number: 15311