Visualizing Optimizers using Chebyshev Proxies and Fatou Sets

Rene Winchenbach, Nils Thuerey

Abstract: With recent advances in optimization many different optimization approaches have been proposed, especially regarding the optimization of weights for neural networks. However, comparing these approaches in a visually succinct and intuitive manner is difficult to do, especially without relying on simplified toy examples that may not be representative. In this paper, we present a visualization toolkit using a modified variant of Fatou sets of functions in the complex domain to directly visualize the convergence behavior of an optimizer across a large range of input values. Furthermore, we propose an approach of generating test functions based on polynomial Chebyshev proxies, with polynomial degrees up to 11217, and a modification of these proxies to yield functions that are strictly positive with known global minima, i.e., roots. Our proposed toolkit is provided as a cross platform open source framework in C++ using OpenMP for parallelization. Finally, for menomorphic functions the process generates visually interesting fractals, which might also be interesting from an artistic standpoint.