|
| | LBFGSBSolver (int numberOfVariables) |
| | Creates a new instance of the L-BFGS optimization algorithm. More...
|
| |
| | LBFGSBSolver (int numberOfVariables, std::function< Scalar(const af::array &, af::array &)> function) |
| | Creates a new instance of the L-BFGS optimization algorithm. More...
|
| |
| | LBFGSBSolver (NonlinearObjectiveFunction< Scalar > *function) |
| | Creates a new instance of the L-BFGS optimization algorithm. More...
|
| |
|
void | SetHistorySize (const int hs) |
| |
| Scalar | GetTolerance () |
| | Gets the relative difference threshold to be used as stopping criteria between two iterations. Default is 0 (iterate until convergence). More...
|
| |
| void | SetTolerance (Scalar tolerance) |
| | Sets the relative difference threshold to be used as stopping criteria between two iterations. Default is 0 (iterate until convergence). More...
|
| |
| int | GetMaxIterations () |
| | Gets the maximum number of iterations to be performed during optimization. Default is 0 (iterate until convergence). More...
|
| |
| void | SetMaxIterations (int iter) |
| | Sets the maximum number of iterations to be performed during optimization. Default is 0 (iterate until convergence). More...
|
| |
| int | GetIterations () |
| | Gets the number of iterations performed in the last call to IOptimizationMethod.Minimize(). More...
|
| |
| virtual int | GetNumberOfVariables () |
| | Gets the number of variables (free parameters) in the optimization problem. More...
|
| |
| virtual af::array | GetSolution () |
| | Gets the current solution found, the values of the parameters which optimizes the function. More...
|
| |
| virtual void | SetSolution (af::array &x) |
| | Sets the current solution found, the values of the parameters which optimizes the function. More...
|
| |
| virtual Scalar | GetValue () |
| | Gets the output of the function at the current Solution. More...
|
| |
| virtual bool | Maximize (af::array &values, int *cycle=nullptr) |
| | Finds the maximum value of a function. The solution vector will be made available at the Solution property. More...
|
| |
| virtual bool | Minimize (af::array &values, int *cycle=nullptr) |
| | Finds the minimum value of a function. The solution vector will be made available at the Solution property. More...
|
| |
| virtual bool | Maximize (int *cycle=nullptr) |
| | Finds the maximum value of a function. The solution vector will be made available at the Solution property. More...
|
| |
| virtual bool | Minimize (int *cycle=nullptr) |
| | Finds the minimum value of a function. The solution vector will be made available at the Solution property. More...
|
| |
| void | Display (bool display) |
| | Set to display optimization information. More...
|
| |
| virtual int | GetNumberOfVariables ()=0 |
| | Gets the number of variables (free parameters) in the optimization problem. More...
|
| |
| virtual af::array | GetSolution ()=0 |
| | Gets the current solution found, the values of the parameters which optimizes the function. More...
|
| |
| virtual void | SetSolution (af::array &x)=0 |
| | Gets a solution. More...
|
| |
| virtual Scalar | GetValue ()=0 |
| | Gets the output of the function at the current Solution. More...
|
| |
| virtual bool | Minimize (int *cycle=nullptr)=0 |
| | Finds the minimum value of a function. The solution vector will be made available at the Solution property. More...
|
| |
| virtual bool | Maximize (int *cycle=nullptr)=0 |
| | Finds the maximum value of a function. The solution vector will be made available at the Solution property. More...
|
| |
|
| virtual bool | Optimize (int *cycle=nullptr) override |
| | Implements the actual optimization algorithm. This method should try to minimize the objective function. More...
|
| |
| std::vector< int > | SortIndexes (const std::vector< std::pair< int, Scalar > > &v) |
| | Sorts pairs (k,v) according v ascending. More...
|
| |
| void | GetGeneralizedCauchyPoint (const af::array &x, const af::array &g, af::array &x_cauchy, af::array &c) |
| | Computation of the generalized Cauchy point. More...
|
| |
| Scalar | FindAlpha (af::array &x_cp, af::array &du, std::vector< int > &FreeVariables) |
| | Finds alpha* = max{a : a <= 1 and l_i-xc_i <= a*d_i <= u_i-xc_i}. More...
|
| |
| void | SubspaceMinimization (af::array &x_cauchy, af::array &x, af::array &c, af::array &g, af::array &SubspaceMin) |
| | Solving unbounded probelm. More...
|
| |
| Scalar | GetOptimality (const af::array &x, const af::array &g) |
| | Gets an optimality. More...
|
| |
| | BaseGradientOptimizationMethod (int numberOfVariables) |
| | Initializes a new instance of the BaseGradientOptimizationMethod class. More...
|
| |
| | BaseGradientOptimizationMethod (int numberOfVariables, std::function< Scalar(const af::array &, af::array &)> function) |
| | Initializes a new instance of the BaseGradientOptimizationMethod class. More...
|
| |
| | BaseGradientOptimizationMethod (NonlinearObjectiveFunction< Scalar > *function) |
| | Initializes a new instance of the BaseGradientOptimizationMethod class. More...
|
| |
| void | InitLinesearch () |
| | Inits linesearch. More...
|
| |
| void | SetValue (Scalar v) |
| | Sets the output of the function at the current Solution. More...
|
| |
| void | SetNumberOfVariables (int n) |
| | Sets the number of variables (free parameters) in the optimization problem. More...
|
| |
| | BaseOptimizationMethod (int numberOfVariables) |
| | Initializes a new instance of the BaseOptimizationMethod class. More...
|
| |
| | BaseOptimizationMethod (int numberOfVariables, std::function< Scalar(const af::array &, af::array &)> function) |
| | Initializes a new instance of the BaseOptimizationMethod class. More...
|
| |
| | BaseOptimizationMethod (NonlinearObjectiveFunction< Scalar > *function) |
| | Initializes a new instance of the BaseOptimizationMethod class. More...
|
| |
| virtual bool | Optimize (int *cycle=nullptr)=0 |
| | Implements the actual optimization algorithm. This method should try to minimize the objective function. More...
|
| |
template<typename
Scalar, LineSearchType LSType = MoreThuente>
class NeuralEngine::MachineLearning::LBFGSBSolver< Scalar, LSType >
Limited-memory BFGS (L-BFGS or LM-BFGS).
Limited-memory BFGS (L-BFGS or LM-BFGS) is an optimization algorithm in the family of quasi-Newton methods that approximates the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm using a limited amount of computer memory. It is a popular algorithm for parameter estimation in machine learning. The algorithm's target problem is to minimize $f ( x )$ $f(\mathbf {x} )$ over unconstrained values of the real-vector $\mathbf{x}$ where $f$ is a differentiable scalar function.
Like the original BFGS, L-BFGS uses an estimation to the inverse Hessian matrix to steer its search through variable space, but where BFGS stores a dense $n\times n$ approximation to the inverse Hessian (n being the number of variables in the problem), L-BFGS stores only a few vectors that represent the approximation implicitly. Due to its resulting linear memory requirement, the L-BFGS method is particularly well suited for optimization problems with a large number of variables. Instead of the inverse Hessian $\mathbf{H}_k$, L-BFGS maintains a history of the past m updates of the position $\mathbf{x}$ and gradient ∇f(x), where generally the history size $m$ can be small (often $m<10$). These updates are used to implicitly do operations requiring the $\mathbf{H}_k$-vector product.
References:
HmetalT, 02.05.2019.
Definition at line 76 of file FgLBFGSBsolver.h.