|
| LBFGSBSolver (int numberOfVariables) |
| Creates a new instance of the L-BFGS optimization algorithm. More...
|
|
| LBFGSBSolver (int numberOfVariables, std::function< Scalar(const af::array &, af::array &)> function) |
| Creates a new instance of the L-BFGS optimization algorithm. More...
|
|
| LBFGSBSolver (NonlinearObjectiveFunction< Scalar > *function) |
| Creates a new instance of the L-BFGS optimization algorithm. More...
|
|
void | SetHistorySize (const int hs) |
|
Scalar | GetTolerance () |
| Gets the relative difference threshold to be used as stopping criteria between two iterations. Default is 0 (iterate until convergence). More...
|
|
void | SetTolerance (Scalar tolerance) |
| Sets the relative difference threshold to be used as stopping criteria between two iterations. Default is 0 (iterate until convergence). More...
|
|
int | GetMaxIterations () |
| Gets the maximum number of iterations to be performed during optimization. Default is 0 (iterate until convergence). More...
|
|
void | SetMaxIterations (int iter) |
| Sets the maximum number of iterations to be performed during optimization. Default is 0 (iterate until convergence). More...
|
|
int | GetIterations () |
| Gets the number of iterations performed in the last call to IOptimizationMethod.Minimize(). More...
|
|
virtual int | GetNumberOfVariables () |
| Gets the number of variables (free parameters) in the optimization problem. More...
|
|
virtual af::array | GetSolution () |
| Gets the current solution found, the values of the parameters which optimizes the function. More...
|
|
virtual void | SetSolution (af::array &x) |
| Sets the current solution found, the values of the parameters which optimizes the function. More...
|
|
virtual Scalar | GetValue () |
| Gets the output of the function at the current Solution. More...
|
|
virtual bool | Maximize (af::array &values, int *cycle=nullptr) |
| Finds the maximum value of a function. The solution vector will be made available at the Solution property. More...
|
|
virtual bool | Minimize (af::array &values, int *cycle=nullptr) |
| Finds the minimum value of a function. The solution vector will be made available at the Solution property. More...
|
|
virtual bool | Maximize (int *cycle=nullptr) |
| Finds the maximum value of a function. The solution vector will be made available at the Solution property. More...
|
|
virtual bool | Minimize (int *cycle=nullptr) |
| Finds the minimum value of a function. The solution vector will be made available at the Solution property. More...
|
|
void | Display (bool display) |
| Set to display optimization information. More...
|
|
virtual int | GetNumberOfVariables ()=0 |
| Gets the number of variables (free parameters) in the optimization problem. More...
|
|
virtual af::array | GetSolution ()=0 |
| Gets the current solution found, the values of the parameters which optimizes the function. More...
|
|
virtual void | SetSolution (af::array &x)=0 |
| Gets a solution. More...
|
|
virtual Scalar | GetValue ()=0 |
| Gets the output of the function at the current Solution. More...
|
|
virtual bool | Minimize (int *cycle=nullptr)=0 |
| Finds the minimum value of a function. The solution vector will be made available at the Solution property. More...
|
|
virtual bool | Maximize (int *cycle=nullptr)=0 |
| Finds the maximum value of a function. The solution vector will be made available at the Solution property. More...
|
|
|
virtual bool | Optimize (int *cycle=nullptr) override |
| Implements the actual optimization algorithm. This method should try to minimize the objective function. More...
|
|
std::vector< int > | SortIndexes (const std::vector< std::pair< int, Scalar > > &v) |
| Sorts pairs (k,v) according v ascending. More...
|
|
void | GetGeneralizedCauchyPoint (const af::array &x, const af::array &g, af::array &x_cauchy, af::array &c) |
| Computation of the generalized Cauchy point. More...
|
|
Scalar | FindAlpha (af::array &x_cp, af::array &du, std::vector< int > &FreeVariables) |
| Finds alpha* = max{a : a <= 1 and l_i-xc_i <= a*d_i <= u_i-xc_i}. More...
|
|
void | SubspaceMinimization (af::array &x_cauchy, af::array &x, af::array &c, af::array &g, af::array &SubspaceMin) |
| Solving unbounded probelm. More...
|
|
Scalar | GetOptimality (const af::array &x, const af::array &g) |
| Gets an optimality. More...
|
|
| BaseGradientOptimizationMethod (int numberOfVariables) |
| Initializes a new instance of the BaseGradientOptimizationMethod class. More...
|
|
| BaseGradientOptimizationMethod (int numberOfVariables, std::function< Scalar(const af::array &, af::array &)> function) |
| Initializes a new instance of the BaseGradientOptimizationMethod class. More...
|
|
| BaseGradientOptimizationMethod (NonlinearObjectiveFunction< Scalar > *function) |
| Initializes a new instance of the BaseGradientOptimizationMethod class. More...
|
|
void | InitLinesearch () |
| Inits linesearch. More...
|
|
void | SetValue (Scalar v) |
| Sets the output of the function at the current Solution. More...
|
|
void | SetNumberOfVariables (int n) |
| Sets the number of variables (free parameters) in the optimization problem. More...
|
|
| BaseOptimizationMethod (int numberOfVariables) |
| Initializes a new instance of the BaseOptimizationMethod class. More...
|
|
| BaseOptimizationMethod (int numberOfVariables, std::function< Scalar(const af::array &, af::array &)> function) |
| Initializes a new instance of the BaseOptimizationMethod class. More...
|
|
| BaseOptimizationMethod (NonlinearObjectiveFunction< Scalar > *function) |
| Initializes a new instance of the BaseOptimizationMethod class. More...
|
|
virtual bool | Optimize (int *cycle=nullptr)=0 |
| Implements the actual optimization algorithm. This method should try to minimize the objective function. More...
|
|
template<typename
Scalar, LineSearchType LSType = MoreThuente>
class NeuralEngine::MachineLearning::LBFGSBSolver< Scalar, LSType >
Limited-memory BFGS (L-BFGS or LM-BFGS).
Limited-memory BFGS (L-BFGS or LM-BFGS) is an optimization algorithm in the family of quasi-Newton methods that approximates the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm using a limited amount of computer memory. It is a popular algorithm for parameter estimation in machine learning. The algorithm's target problem is to minimize $f ( x )$ $f(\mathbf {x} )$ over unconstrained values of the real-vector $\mathbf{x}$ where $f$ is a differentiable scalar function.
Like the original BFGS, L-BFGS uses an estimation to the inverse Hessian matrix to steer its search through variable space, but where BFGS stores a dense $n\times n$ approximation to the inverse Hessian (n being the number of variables in the problem), L-BFGS stores only a few vectors that represent the approximation implicitly. Due to its resulting linear memory requirement, the L-BFGS method is particularly well suited for optimization problems with a large number of variables. Instead of the inverse Hessian $\mathbf{H}_k$, L-BFGS maintains a history of the past m updates of the position $\mathbf{x}$ and gradient ∇f(x), where generally the history size $m$ can be small (often $m<10$). These updates are used to implicitly do operations requiring the $\mathbf{H}_k$-vector product.
References:
HmetalT, 02.05.2019.
Definition at line 76 of file FgLBFGSBsolver.h.