Error analysis of numerical methods for optimization problems

Alexey Chernov, Anna Lisachenko, Julia Chernova

Keywords: Solution error estimates, upper and lower error estimates, exact estimates of the error of solving the optimization problem, three-point scheme in the method of estimating the error of the solution, rounding method of the solution error estimation

TL;DR: The article considers methods of constructing solution error estimates in optimization problems

Abstract:

The article considers methods of constructing solution error estimates in optimization problems. Estimates of the solution error can be divided into two classes: theoretical and numerical. Estimates of the solution error belonging to the first class are calculated on the basis of the theoretical analysis of the convergence of the problem solution method. The calculated value of error in this case is a function of the parameters of the problem and its solution method whose values very often do not seem possible to determine. Therefore theoretical estimates of the solution error often appear useful only for the qualitative analysis of the numerical method convergence. Estimates of solution error belonging to the second class are known only for a small number of methods of solution of optimization problems: finite-step simplex method for solution of linear programming problems, many zero-order methods for solution of one-dimensional minimization problems, interval estimates for solution of direct and dual problems of mathematical programming. In this paper we propose two new methods of constructing numerical estimates of error, in which the error estimate is a known function of values calculated in the process of solving the optimization problem by some method, the numerical value of which is calculated explicitly. The class of optimization problems and the class of their solution methods for which the new methods are defined is wide enough: the objective function is defined on some closed set $n$-dimensional Euclidean space and is continuous on it; the method of the solution of the optimization problem is defined by monotonically decreasing convergent sequence of values of the objective function.

The first proposed method for estimating the error of the solution is based on the so-called three-point scheme. A group of three elements is singled out from the decreasing sequence of function values, i.e. function values whose ratio of consecutive deviations is less than unity. The formula for estimating the error of the solution is specified for such points. Assuming in this scheme two (extreme) points to be fixed and the average point to be variable, the formula for exact evaluation of the decision error is obtained, which, however, depends also on the optimal value of the target function. Using the value of the function calculated at a finite step instead, an estimate of the solution error is obtained which can be sufficiently accurate.

The second proposed method is called the rounding method of the solution error estimation. This method is based on the assumption that the value of the target function increases the number of exact digits in the decimal representation of the optimal solution of the optimization problem when the number of iteration increases. This condition makes it possible to find an estimate of solution error for each iteration of the numerical method. Value of the target function in this method consists of some number of digits corresponding to the rounding of the optimal solution of the problem.

The paper presents the results of numerical experiment on estimation of the solution error for the values of the target function and the norm of the argument value as their deviations from the optimal values.

Submission Number: 66