Function information

optimistix.FunctionInfo

Different solvers (BFGS, Levenberg--Marquardt, ...) evaluate different quantities of the objective function. Some may compute gradient information, some may provide approximate Hessian information, etc.

This enumeration-ish object captures the different variants.

Available variants are optimistix.FunctionInfo.{Eval, EvalGrad, EvalGradHessian, EvalGradHessianInv, Residual, ResidualJac}.

as_min(self) -> Array abstractmethod

For a minimisation problem, returns f(y). For a least-squares problem, returns 0.5*residuals^2 -- i.e. its loss as a minimisation problem.

optimistix.FunctionInfo.Eval (FunctionInfo)

Has a .f attribute describing fn(y). Used when no gradient information is available.

optimistix.FunctionInfo.EvalGrad (FunctionInfo)

Has a .f attribute as with optimistix.FunctionInfo.Eval. Also has a .grad attribute describing d(fn)/dy. Used with first-order solvers for minimisation problems. (E.g. gradient descent; nonlinear CG.)

optimistix.FunctionInfo.EvalGradHessian (FunctionInfo)

Has .f and .grad attributes as with optimistix.FunctionInfo.EvalGrad. Also has a .hessian attribute describing (an approximation to) the Hessian of fn at y. Used with quasi-Newton minimisation algorithms, like BFGS.

optimistix.FunctionInfo.EvalGradHessianInv (FunctionInfo)

As optimistix.FunctionInfo.EvalGradHessian, but records the (approximate) inverse-Hessian instead. Has .f and .grad and .hessian_inv attributes.

optimistix.FunctionInfo.Residual (FunctionInfo)

Has a .residual attribute describing fn(y). Used with least squares problems, for which fn returns residuals.

optimistix.FunctionInfo.ResidualJac (FunctionInfo)

Records the Jacobian d(fn)/dy as a linear operator. Used for least squares problems, for which fn returns residuals. Has .residual and .jac and .grad attributes, where residual = fn(y), jac = d(fn)/dy and grad = jac^T residual.

Takes just residual and jac as __init__-time arguments, from which grad is computed.