A New Efficient Method For Combining Gradients Of Different Orders

ICLR 2026 Conference Submission25194 Authors

Keywords: gradient method, gradient combine, SD, CBB

Abstract: We present a new optimization method called GOC(Gradient Order Combination) which a combination based on the products of Hessian matrices of different orders and the gradient. the parameter r (the recipprocal of steplenth) is taken as analysis target, we can regard the SD method as a first-order and the CBB method as second-order. We have developed third-order and even higher-order, which offer faster convergence rates.

Primary Area: optimization

Submission Number: 25194