Fast and Accurate Least-Mean-Squares Solvers

Jianchen Zhao, Jiewen Liu, Zhenzhe Zhang

Abstract:

Least-mean squares(LMS) solvers, one of the most elementary supervised learning algorithms, have been heavily used in solving various practical issues due to its high interpretability and nice mathematical closed-form solution. However, in many real world cases, computing the covariance matrix to produce such solution becomes impossible due to some reasons.
In this project, we investigated reproducibility of results from a paper submitted and accepted by 2019 Neural Information Processing Systems (NeurIPS), named Fast and Accurate Least-Mean-Squares Solvers, which proposes a novel method to derive a much smaller but maintainable covariance matrix without accuracy loss, based on a new time-efficient implementation of Caratheodory’s Theorem.
In our study, we first reproduce the tests’ results in the paper and examine the effect of the method. And our experiments on extension of the method reveals some property and limitations in dealing with new cases.

The code can be viewed in: https://github.com/ZZZCal/Reproducibility-of-Fast-and-Accurate-Least-Mean-Squares-Solvers

Track: Ablation

NeurIPS Paper Id: https://openreview.net/forum?id=BkeDpVHlUB