Deep Bootstrap Aggregation via Least Squares Estimation

Chih-Yu Chang, Ming-Chung Chang

Keywords: generalized least squares, bagging, random forest, ensemble learning, regression problems

Abstract:

Bootstrap aggregation, commonly referred to as bagging, is a fundamental technique in ensemble learning designed to enhance the performance of predictive models. It is well-established that the effectiveness of bagging is strongly influenced by the management of correlations among the aggregated models. For instance, random forests, a widely-used ensemble method, address this issue by randomly selecting features to reduce the correlation between individual tree models. In this study, we propose a method called ``Deep Bootstrap Aggregation'' for regression tasks, which combines deep network architectures with least squares estimation to improve the predictive accuracy of bagging models. Both theoretical analysis and empirical experiments support the effectiveness of the proposed approach.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5193