Poblano Toolbox v1.1

Toolbox Overview

Poblano is a toolbox of large-scale algorithms for nonlinear optimization. The algorithms in Poblano require only first-order derivative information (e.g., gradients for scalar-valued objective functions).

Contents


Introduction

Poblano optimizers find local minimizers of scalar-valued objective functions taking vector inputs. Specifically, the problems solved by Poblano optimizers are of the following form:

The gradient of the objective function, , is required for all Poblano optimizers. The optimizers converge to a stationary point, , where

A line search satisfying the strong Wolfe conditions is used to guarantee global convergence of the Poblano optimizers.


Optimization Methods

The following optimization methods are available in Poblano.

Nonlinear conjugate gradient method (ncg) [4]

Limited-memory BFGS method (lbfgs) [4]

Truncated Newton method (tn) [1]


Globalization Strategies

Line search methods


Optimization Input Parameters

Input parameters are passed to the different optimization methods using Matlab inputParser objects. Some parameters are shared across all methods and others are specific to a particular method. Below are descriptions of the shared input parameters and examples of how to set and use these parameters in the optimization methods. The Poblano function poblano_params is used by the optimization methods to set the input parameters. See the Optimization Input Parameters documentation for more details.


Optimization Output Parameters

Each of the optimization methods in Poblano outputs a single structure containing fields for the approximate solution, function and gradient values at the solution, and various information about the optimization run (e.g., number of function evaluations, etc.). The Poblano function poblano_out is used by the optimization methods to set the output parameters. See the Optimization Output Parameters documentation for more details.


Checking Gradient Calculations

Analytic gradients can be checked using finite difference approximations. The Poblano function gradientcheck computes the gradient approximations and compares the results to the analytic gradient using a user-supplied objective function/gradient M-file. The user can choose one of several difference formulas as well as the difference step used in the computations. See the Checking Gradient Calculations documentation for more details.


Examples

Poblano provides two example function/gradient M-files:

See the Poblano Examples documentation for more details.


Calling a Poblano Optimizer

All Poblano methods are called using the name of the method along with two required arguments and one or more optional arguments. The required arguments are 1) a handle to the function being minimized, and 2) the initial guess of the solution (as a scalar or column vector). For example, the following is a call to the ncg method to minimize the example1 function distributed with Poblano starting with an initial guess of and using the default ncg parameters.

ncg(@example1, pi/4);
 Iter  FuncEvals       F(X)          ||G(X)||/N        
------ --------- ---------------- ----------------
     0         1       0.70710678       0.70710678
     1         6      -0.99999998       0.00017407
     2         7      -1.00000000       0.00000000

Parameterize functions can be optimized using Poblano as well. For such functions, the function handle can be used to specify the function parameters. For example, Poblano's example1 function takes an optional scalar parameter as follows.

ncg(@(x) example1(x,3), pi/4);
 Iter  FuncEvals       F(X)          ||G(X)||/N        
------ --------- ---------------- ----------------
     0         1       0.70710678       2.12132034
     1        14      -0.99998885       0.01416497
     2        16      -1.00000000       0.00000147

Functions taking vectors as inputs can be optimized using Poblano as well. For functions which can take input vectors of arbitrary sizes (e.g., Matlab functions such as sin, fft, etc.), the size of the initial guess (as a scalar or column vector) determines the size of the problem to be solved. For example, Poblano's \texttt{example1} function can take as input a vector (in this case a vector in ) as follows.

ncg(@(x) example1(x,3), [pi/5 pi/4 pi/3]');
 Iter  FuncEvals       F(X)          ||G(X)||/N        
------ --------- ---------------- ----------------
     0         1       1.65816330       1.26312767
     1         5      -0.94404964       0.49911677
     2         9      -1.64717576       1.35314598
     3        14      -1.82513979       1.19534779
     4        19      -2.96660154       0.25769184
     5        22      -2.99851295       0.05452749
     6        24      -2.99999996       0.00028818
     7        26      -3.00000000       0.00000000

The optional arguments are input parameters specifying how the optimization method is to be run. See the Input Parameters documentation for details about the input parameters.)


Acknowledgments


References

[1] Dembo, R.S. and and Steihaug, T. (1983). Truncated-Newton algorithms for large-scale unconstrained minimization. Mathematical Programming, 26, 190-212.

[2] Golub, G. H. and Loan, C. F. V. (1996). Matrix Computations. Johns Hopkins University Press.

[3] More, J. J. and Thuente, D. J. (1994). Line search algorithms with guaranteed sufficient decrease. ACM Trans. Math. Softw., 20, 286-307.

[4] Nocedal, J. and Wright S. J. (1999). Numerical Optimization. Springer.


Poblano Toolbox v1.1