Transforming a Non-Differentiable Rasterizer into a Differentiable One with Stochastic Gradient Estimation

Thomas Deliot, Eric Heitz, Laurent Belcour

Keywords: differentiable rendering; stochastic gradient estimation

TL;DR: We turn a non-differentiable rasterizer into a differentiable one with a scalable form of stochastic gradient estimation.

Abstract: We show how to transform a non-differentiable rasterizer into a differentiable one with minimal engineering efforts and no automatic differentiation. To do so, we improve on *Stochastic Gradient Estimation* by using a *Per-Pixel Loss* which leverage the fact that only a few primitives contribute to a given pixel. Estimating gradients on a per-pixel basis bounds the dimensionality of the optimization problem and makes the method scalable. To track parameters contributing to a pixel, we use ID- and UV-buffers, which are often already available or trivial to obtain. With these minor modifications, we obtain an in-engine optimizer for 3D assets with millions of geometry and texture parameters.

Submission Number: 54