Limited memory Broyden Fletcher Goldfarb Shannon minimization. More...
#include <vnl_lbfgs.h>
Public Types | |
| enum | ReturnCodes { ERROR_FAILURE = -1, ERROR_DODGY_INPUT = 0, CONVERGED_FTOL = 1, CONVERGED_XTOL = 2, CONVERGED_XFTOL = 3, CONVERGED_GTOL = 4, FAILED_TOO_MANY_ITERATIONS = 5, TOO_MANY_ITERATIONS = FAILED_TOO_MANY_ITERATIONS, FAILED_FTOL_TOO_SMALL = 6, FAILED_XTOL_TOO_SMALL = 7, FAILED_GTOL_TOO_SMALL = 8, FAILED_USER_REQUEST = 9 } |
| Some generic return codes that apply to all minimizers. More... | |
Public Member Functions | |
| vnl_lbfgs () | |
| Default constructor. | |
| vnl_lbfgs (vnl_cost_function &f) | |
| Constructor. f is the cost function to be minimized. | |
| bool | minimize (vnl_vector< double > &x) |
| void | set_f_tolerance (double v) |
| Set the convergence tolerance on F (sum of squared residuals). | |
| double | get_f_tolerance () const |
| void | set_x_tolerance (double v) |
| Set the convergence tolerance on X. | |
| double | get_x_tolerance () const |
| void | set_g_tolerance (double v) |
| Set the convergence tolerance on Grad(F)' * F. | |
| double | get_g_tolerance () const |
| void | set_max_function_evals (int v) |
| Set the termination maximum number of iterations. | |
| int | get_max_function_evals () const |
| void | set_epsilon_function (double v) |
| Set the step length for FD Jacobian. | |
| double | get_epsilon_function () const |
| void | set_trace (bool on) |
| Turn on per-iteration printouts. | |
| bool | get_trace () const |
| void | set_verbose (bool verb) |
| Set verbose flag. | |
| bool | get_verbose () const |
| void | set_check_derivatives (int cd) |
| Set check_derivatives flag. Negative values may mean fewer checks. | |
| int | get_check_derivatives () const |
| double | get_start_error () const |
| Return the error of the function when it was evaluated at the start point of the last minimization. | |
| double | get_end_error () const |
| Return the best error that was achieved by the last minimization, corresponding to the returned x. | |
| int | get_num_evaluations () const |
| Return the total number of times the function was evaluated by the last minimization. | |
| int | get_num_iterations () const |
| Return the number of {iterations} in the last minimization. | |
| bool | obj_value_reduced () |
| Whether the error reduced in the last minimization. | |
| virtual vnl_matrix< double > const & | get_covariance () |
| Return the covariance of the estimate at the end. | |
| virtual vcl_string | is_a () const |
| Return the name of the class. | |
| virtual bool | is_class (vcl_string const &s) const |
| Return true if the name of the class matches the argument. | |
| ReturnCodes | get_failure_code () const |
| Return the failure code of the last minimization. | |
Public Attributes | |
| int | memory |
| Step accuracy/speed tradeoff. | |
| double | line_search_accuracy |
| Accuracy of line search. | |
| double | default_step_length |
| Default step length in line search. | |
Protected Member Functions | |
| void | reset () |
| void | report_eval (double f) |
| Called by derived classes after each function evaluation. | |
| virtual bool | report_iter () |
| Called by derived classes after each iteration. | |
Protected Attributes | |
| double | xtol |
| Termination tolerance on X (solution vector) | |
| long | maxfev |
| Termination maximum number of iterations. | |
| double | ftol |
| Termination tolerance on F (sum of squared residuals) | |
| double | gtol |
| Termination tolerance on Grad(F)' * F = 0. | |
| double | epsfcn |
| Step length for FD Jacobian. | |
| unsigned | num_iterations_ |
| long | num_evaluations_ |
| double | start_error_ |
| double | end_error_ |
| bool | trace |
| bool | verbose_ |
| int | check_derivatives_ |
| ReturnCodes | failure_code_ |
Private Member Functions | |
| void | init_parameters () |
| Called by constructors. | |
Private Attributes | |
| vnl_cost_function * | f_ |
Detailed Description
Limited memory Broyden Fletcher Goldfarb Shannon minimization.
Considered to be the best optimisation algorithm for functions which are well behaved (i.e. locally smooth without too many local minima,) have 1st derivatives available, and do not have a structure that makes them suitable for alternative methods (e.g. if f(x) is a sum of squared terms you should use vnl_levenberg_marquardt.)
This limited-memory rank-2 quasi-newton method maintains an estimate of (the inverse of) the Hessian matrix of f at x. Unlike Newton's method, it doesn't need 2nd derivatives of f(x), has superlinear rather than quadratic convergence and is better behaved away from minima. 2 ranks of this matrix are updated at each step. In order to reduce memory and time requirements, this limited memory version of BFGS only maintains a certain number of vector corrections to a diagonal estimate of the inverse Hessian estimate.
Definition at line 41 of file vnl_lbfgs.h.
Member Enumeration Documentation
enum vnl_nonlinear_minimizer::ReturnCodes [inherited] |
Some generic return codes that apply to all minimizers.
- Enumerator:
Definition at line 102 of file vnl_nonlinear_minimizer.h.
Constructor & Destructor Documentation
| vnl_lbfgs::vnl_lbfgs | ( | ) |
Default constructor.
memory is set to 5, line_search_accuracy to 0.9. Calls init_parameters
Definition at line 22 of file vnl_lbfgs.cxx.
| vnl_lbfgs::vnl_lbfgs | ( | vnl_cost_function & | f | ) |
Constructor. f is the cost function to be minimized.
Calls init_parameters
Definition at line 30 of file vnl_lbfgs.cxx.
Member Function Documentation
| int vnl_nonlinear_minimizer::get_check_derivatives | ( | ) | const [inline, inherited] |
Definition at line 82 of file vnl_nonlinear_minimizer.h.
| vnl_matrix< double > const & vnl_nonlinear_minimizer::get_covariance | ( | ) | [virtual, inherited] |
Return the covariance of the estimate at the end.
Definition at line 35 of file vnl_nonlinear_minimizer.cxx.
| double vnl_nonlinear_minimizer::get_end_error | ( | ) | const [inline, inherited] |
Return the best error that was achieved by the last minimization, corresponding to the returned x.
Definition at line 92 of file vnl_nonlinear_minimizer.h.
| double vnl_nonlinear_minimizer::get_epsilon_function | ( | ) | const [inline, inherited] |
Definition at line 70 of file vnl_nonlinear_minimizer.h.
| double vnl_nonlinear_minimizer::get_f_tolerance | ( | ) | const [inline, inherited] |
Definition at line 46 of file vnl_nonlinear_minimizer.h.
| ReturnCodes vnl_nonlinear_minimizer::get_failure_code | ( | ) | const [inline, inherited] |
Return the failure code of the last minimization.
Definition at line 132 of file vnl_nonlinear_minimizer.h.
| double vnl_nonlinear_minimizer::get_g_tolerance | ( | ) | const [inline, inherited] |
Definition at line 60 of file vnl_nonlinear_minimizer.h.
| int vnl_nonlinear_minimizer::get_max_function_evals | ( | ) | const [inline, inherited] |
Definition at line 64 of file vnl_nonlinear_minimizer.h.
| int vnl_nonlinear_minimizer::get_num_evaluations | ( | ) | const [inline, inherited] |
Return the total number of times the function was evaluated by the last minimization.
Definition at line 95 of file vnl_nonlinear_minimizer.h.
| int vnl_nonlinear_minimizer::get_num_iterations | ( | ) | const [inline, inherited] |
Return the number of {iterations} in the last minimization.
Each iteration may have comprised several function evaluations.
Definition at line 99 of file vnl_nonlinear_minimizer.h.
| double vnl_nonlinear_minimizer::get_start_error | ( | ) | const [inline, inherited] |
Return the error of the function when it was evaluated at the start point of the last minimization.
For minimizers driven by a vnl_least_squares_function (Levenberg-Marquardt) this is usually the RMS error. For those driven by a vnl_cost_function (CG, LBFGS, Amoeba) it is simply the value of the vnl_cost_function at the start (usually the sum of squared residuals).
Definition at line 89 of file vnl_nonlinear_minimizer.h.
| bool vnl_nonlinear_minimizer::get_trace | ( | ) | const [inline, inherited] |
Definition at line 74 of file vnl_nonlinear_minimizer.h.
| bool vnl_nonlinear_minimizer::get_verbose | ( | ) | const [inline, inherited] |
Definition at line 78 of file vnl_nonlinear_minimizer.h.
| double vnl_nonlinear_minimizer::get_x_tolerance | ( | ) | const [inline, inherited] |
Definition at line 56 of file vnl_nonlinear_minimizer.h.
| void vnl_lbfgs::init_parameters | ( | ) | [private] |
Called by constructors.
Memory is set to 5, line_search_accuracy to 0.9, default_step_length to 1.
Definition at line 38 of file vnl_lbfgs.cxx.
| vcl_string vnl_nonlinear_minimizer::is_a | ( | ) | const [virtual, inherited] |
Return the name of the class.
Used by polymorphic IO
Definition at line 75 of file vnl_nonlinear_minimizer.cxx.
| bool vnl_nonlinear_minimizer::is_class | ( | vcl_string const & | s | ) | const [virtual, inherited] |
Return true if the name of the class matches the argument.
Used by polymorphic IO
Definition at line 83 of file vnl_nonlinear_minimizer.cxx.
| bool vnl_lbfgs::minimize | ( | vnl_vector< double > & | x | ) |
Definition at line 45 of file vnl_lbfgs.cxx.
| bool vnl_nonlinear_minimizer::obj_value_reduced | ( | ) | [inline, inherited] |
Whether the error reduced in the last minimization.
Definition at line 118 of file vnl_nonlinear_minimizer.h.
| void vnl_nonlinear_minimizer::report_eval | ( | double | f | ) | [protected, inherited] |
Called by derived classes after each function evaluation.
Definition at line 50 of file vnl_nonlinear_minimizer.cxx.
| bool vnl_nonlinear_minimizer::report_iter | ( | ) | [protected, virtual, inherited] |
Called by derived classes after each iteration.
When true is returned, minimizer should stop with code FAILED_USER_REQUEST. Derived classes can redefine this function to make the optimizer stop when a condition is satisfied.
Definition at line 63 of file vnl_nonlinear_minimizer.cxx.
| void vnl_nonlinear_minimizer::reset | ( | ) | [protected, inherited] |
Definition at line 41 of file vnl_nonlinear_minimizer.cxx.
| void vnl_nonlinear_minimizer::set_check_derivatives | ( | int | cd | ) | [inline, inherited] |
Set check_derivatives flag. Negative values may mean fewer checks.
Definition at line 81 of file vnl_nonlinear_minimizer.h.
| void vnl_nonlinear_minimizer::set_epsilon_function | ( | double | v | ) | [inline, inherited] |
Set the step length for FD Jacobian.
Be aware that set_x_tolerance will reset this to xtol * 0.001. The default is 1e-11.
Definition at line 69 of file vnl_nonlinear_minimizer.h.
| void vnl_nonlinear_minimizer::set_f_tolerance | ( | double | v | ) | [inline, inherited] |
Set the convergence tolerance on F (sum of squared residuals).
When the differences in successive RMS errors is less than this, the routine terminates. So this is effectively the desired precision of your minimization. Setting it too low wastes time, too high might cause early convergence. The default of 1e-9 is on the safe side, but if speed is an issue, you can try raising it.
Definition at line 45 of file vnl_nonlinear_minimizer.h.
| void vnl_nonlinear_minimizer::set_g_tolerance | ( | double | v | ) | [inline, inherited] |
Set the convergence tolerance on Grad(F)' * F.
Definition at line 59 of file vnl_nonlinear_minimizer.h.
| void vnl_nonlinear_minimizer::set_max_function_evals | ( | int | v | ) | [inline, inherited] |
Set the termination maximum number of iterations.
Definition at line 63 of file vnl_nonlinear_minimizer.h.
| void vnl_nonlinear_minimizer::set_trace | ( | bool | on | ) | [inline, inherited] |
Turn on per-iteration printouts.
Definition at line 73 of file vnl_nonlinear_minimizer.h.
| void vnl_nonlinear_minimizer::set_verbose | ( | bool | verb | ) | [inline, inherited] |
Set verbose flag.
Definition at line 77 of file vnl_nonlinear_minimizer.h.
| void vnl_nonlinear_minimizer::set_x_tolerance | ( | double | v | ) | [inline, inherited] |
Set the convergence tolerance on X.
When the length of the steps taken in X are about this long, the routine terminates. The default is 1e-8, which should work for many problems, but if you can get away with 1e-4, say, minimizations will be much quicker.
Definition at line 52 of file vnl_nonlinear_minimizer.h.
Member Data Documentation
int vnl_nonlinear_minimizer::check_derivatives_ [protected, inherited] |
Definition at line 153 of file vnl_nonlinear_minimizer.h.
Default step length in line search.
If, on tracing, the STP is always 1, then you could try setting this to a higher value to see how far along the gradient the minimum typically is. Then set this to a number just below that to get maximally far with the single evaluation.
Definition at line 69 of file vnl_lbfgs.h.
double vnl_nonlinear_minimizer::end_error_ [protected, inherited] |
Definition at line 147 of file vnl_nonlinear_minimizer.h.
double vnl_nonlinear_minimizer::epsfcn [protected, inherited] |
Step length for FD Jacobian.
Definition at line 141 of file vnl_nonlinear_minimizer.h.
vnl_cost_function* vnl_lbfgs::f_ [private] |
Definition at line 73 of file vnl_lbfgs.h.
ReturnCodes vnl_nonlinear_minimizer::failure_code_ [protected, inherited] |
Definition at line 154 of file vnl_nonlinear_minimizer.h.
double vnl_nonlinear_minimizer::ftol [protected, inherited] |
Termination tolerance on F (sum of squared residuals)
Definition at line 139 of file vnl_nonlinear_minimizer.h.
double vnl_nonlinear_minimizer::gtol [protected, inherited] |
Termination tolerance on Grad(F)' * F = 0.
Definition at line 140 of file vnl_nonlinear_minimizer.h.
Accuracy of line search.
If function evaluations are cheap wrt the actual minimization steps, change to 0.1, from default of 0.9;
Definition at line 62 of file vnl_lbfgs.h.
long vnl_nonlinear_minimizer::maxfev [protected, inherited] |
Termination maximum number of iterations.
Definition at line 138 of file vnl_nonlinear_minimizer.h.
Step accuracy/speed tradeoff.
Effectively the number of correction vectors to the diagonal approximation of the inverse Hessian estimate that are kept.
Large values of M will result in excessive computing time. 3<= memory <=7 is recommended. Memory requirements will be roughly Const+sizeof(element)*dim(X)*memory. Default is 5.
Definition at line 57 of file vnl_lbfgs.h.
long vnl_nonlinear_minimizer::num_evaluations_ [protected, inherited] |
Definition at line 145 of file vnl_nonlinear_minimizer.h.
unsigned vnl_nonlinear_minimizer::num_iterations_ [protected, inherited] |
Definition at line 144 of file vnl_nonlinear_minimizer.h.
double vnl_nonlinear_minimizer::start_error_ [protected, inherited] |
Definition at line 146 of file vnl_nonlinear_minimizer.h.
bool vnl_nonlinear_minimizer::trace [protected, inherited] |
Definition at line 149 of file vnl_nonlinear_minimizer.h.
bool vnl_nonlinear_minimizer::verbose_ [protected, inherited] |
Definition at line 152 of file vnl_nonlinear_minimizer.h.
double vnl_nonlinear_minimizer::xtol [protected, inherited] |
Termination tolerance on X (solution vector)
Definition at line 137 of file vnl_nonlinear_minimizer.h.
The documentation for this class was generated from the following files:
- core/vnl/algo/vnl_lbfgs.h
- core/vnl/algo/vnl_lbfgs.cxx
