Remember to correct the bias when using deep learning for regression!
Abstract: When training deep learning models for least-squares regression, we cannot expect that the training error residuals of the final model, selected after a fixed training time or based on performance on a hold-out data set, sum to zero. This can introduce a systematic error that accumulates if we are interested in the total aggregated performance over many data points. We suggest adjusting the bias of the machine learning model after training as a default post-processing step, which efficiently solves the problem. The severeness of the error accumulation and the effectiveness of the bias correction are demonstrated in exemplary experiments.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: We carefully revised the manuscript based on the comments from the reviewers. All relevant changes are highlighted in blue in the revised manuscript. The changes are also described in detail in our answers to the reviewers.
Update May 3: Corrected an insignificant typo in eq. (7).
Update May 24: Formatting error corrected
Assigned Action Editor: ~Cedric_Archambeau1
Submission Number: 21
Loading