Randomized iterations for low latency fixed point computation

Neal Master, Nicholas Bambos

Abstract: Many algorithms in numerical analysis, operations research, and control theory can be written as fixed point iterations of contraction maps. Because these algorithms are so pervasive, there is significant value in being able to perform these computations with lower computational latency. Previous work has focused on reducing the computational complexity of certain algorithms, but there has been less of a focus on using massively parallel computing systems to reduce computational latency. The rise of parallel computing systems makes this focus increasingly relevant as many traditional algorithms are incapable of fully utilizing such large-scale parallel processing power. We propose a randomized parallel algorithm which computes the fixed point of an arbitrary contraction map on ℝ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> while making full use of large scale computing resources. When the number of processors grows exponentially with n, the proposed algorithm allows for a linear reduction in latency. Though this can also be said of a naive “brute-force” algorithm, the proposed algorithm is characterized by a much better linear factor. A numerical example is used to demonstrate this latency reduction while analytical proofs show this improvement holds in general. We conclude by discussing potential future work in specializing the proposed algorithm for specific applications as well in building a more general theory.