The cost function for an optimization routine. More...

#include <CostFunction.h>

Inheritance diagram for muq::Optimization::CostFunction:

Detailed Description

The cost function for an optimization routine.

The cost function has the form:

\begin{equation} c = J(x), \end{equation}

where, \(c \in \mathbb{R}\).

This class defines an interface for evaluating such a cost function, as well as evaluating the gradient and Hessian. There are two ways to use this class. The first, is just to call the Cost, Gradient, and Hessian functions with the point we want to evaluate. This is shown below:

std::shared_ptr<CostFunction> cost = ...
 
Eigen::VectorXd evalPt = ...
 
// Compute the cost
double c = cost->Cost(evalPt);
 
// Compute the gradient
Eigen::VectorXd grad = cost->Gradient(evalPt);
 
// Compute the Hessian matrix H
Eigen::MatrixXd hess = cost->Hessian(evalPt);
 
// Compute the matrix product Hg
Eigen::VectorXd hessApp = cost->ApplyHessian(evalPt, grad);

The above code is straightforward, but makes it difficult to share calculations between the Cost, Gradient, and Hessian functions. For example, all of these functions might require the same model evaluation. To help overcome this issue, a second way to use this class is by first setting the evaluation point and then calling the Cost, Gradient, or Hessian functions. For example,

std::shared_ptr<CostFunction> cost = ...
 
Eigen::VectorXd evalPt = ...
 
// Set the shared evaluation point
cost->SetPoint(evalPt);
 
// Extract the cost
double c = cost->Cost();
 
// Compute the gradient
Eigen::VectorXd grad = cost->Gradient();
 
// Compute the Hessian matrix H
Eigen::MatrixXd hess = cost->Hessian();
 
// Compute the matrix product Hg
Eigen::VectorXd hessApp = cost->ApplyHessian(grad);

By setting the evaluation point first, we can precompute any information that might be shared across calls to Cost, Gradient, Hessian, etc...

To define a child of the CostFunction class, the user needs to implement the Cost() function that does not take any arguments. If information can be shared between Cost, Gradient, and Hessian functions, then the user will also want to override the SetPoint function.

Definition at line 74 of file CostFunction.h.

Public Member Functions
	CostFunction (unsigned int dim)

virtual	~CostFunction ()=default

virtual void	SetPoint (Eigen::VectorXd const &evalPt)

virtual double	Cost (Eigen::VectorXd const &x)
	The value of the cost function. More...

virtual double	Cost ()=0

virtual Eigen::VectorXd	Gradient (Eigen::VectorXd const &evalPt)

virtual Eigen::VectorXd	Gradient ()

virtual Eigen::MatrixXd	Hessian (Eigen::VectorXd const &evalPt)
	The Hessian of the cost function. More...

virtual Eigen::MatrixXd	Hessian ()

virtual Eigen::MatrixXd	HessianByFD (Eigen::VectorXd const &evalPt)
	The Hessian of the cost function using finite difference. More...

virtual Eigen::MatrixXd	HessianByFD ()

virtual Eigen::VectorXd	ApplyHessian (Eigen::VectorXd const &evalPt, Eigen::VectorXd const &vec)
	The Hessian of the cost function. More...

virtual Eigen::VectorXd	ApplyHessian (Eigen::VectorXd const &vec)

Public Member Functions inherited from muq::Modeling::ModPiece
	ModPiece (std::vector< int > const &inputSizes, std::vector< int > const &outputSizes)

	ModPiece (Eigen::VectorXi const &inputSizes, Eigen::VectorXi const &outputSizes)

virtual	~ModPiece ()=default

virtual double	GetRunTime (const std::string &method="Evaluate") const override
	Get the average run time for one of the implemented methods. More...

virtual void	ResetCallTime () override
	Resets the number of call and times. More...

virtual unsigned long int	GetNumCalls (const std::string &method="Evaluate") const override
	get the number of times one of the implemented methods has been called. More...

virtual std::vector< Eigen::VectorXd > const &	Evaluate (std::vector< Eigen::VectorXd > const &input)
	Evaluate the ModPiece. More...

virtual std::vector< Eigen::VectorXd > const &	Evaluate (ref_vector< Eigen::VectorXd > const &input)

	VARIADIC_TO_REFVECTOR (Evaluate, Eigen::VectorXd, std::vector< Eigen::VectorXd > const &)

virtual Eigen::VectorXd const &	Gradient (unsigned int const outputDimWrt, unsigned int const inputDimWrt, std::vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sensitivity)
	Compute the Gradient \(J^Tv\). More...

virtual Eigen::VectorXd const &	Gradient (unsigned int const outputDimWrt, unsigned int const inputDimWrt, ref_vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sensitivity)

Eigen::VectorXd const &	Gradient (unsigned int outWrt, unsigned int inWrt, Eigen::VectorXd const &last, Eigen::VectorXd const &sens)

template<typename... Args>
Eigen::VectorXd const &	Gradient (unsigned int wrtOut, unsigned int wrtIn, Args const &... args)

Eigen::VectorXd const &	ApplyHessian (unsigned int outWrt, unsigned int inWrt1, unsigned int inWrt2, Eigen::VectorXd const &last, Eigen::VectorXd const &sens, Eigen::VectorXd const &vec)

template<typename... Args>
Eigen::VectorXd const &	ApplyHessian (unsigned int wrtOut, unsigned int wrtIn1, unsigned int wrtIn2, Args const &... args)

virtual Eigen::MatrixXd const &	Jacobian (unsigned int const outputDimWrt, unsigned int const inputDimWrt, std::vector< Eigen::VectorXd > const &input)
	Compute the Jacobian of this ModPiece. More...

virtual Eigen::MatrixXd const &	Jacobian (unsigned int const outputDimWrt, unsigned int const inputDimWrt, ref_vector< Eigen::VectorXd > const &input)

template<typename... Args>
Eigen::MatrixXd const &	Jacobian (unsigned int outWrt, unsigned int inWrt, Args const &... args)

template<typename... Args>
Eigen::MatrixXd	JacobianByFD (unsigned int outWrt, unsigned int inWrt, Args const &... args)

template<typename... Args>
Eigen::MatrixXd	ApplyJacobianByFD (unsigned int outWrt, unsigned int inWrt, Args const &... args)

virtual Eigen::VectorXd const &	ApplyJacobian (unsigned int const outputDimWrt, unsigned int const inputDimWrt, std::vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &vec)
	Apply the Jacobian of this ModPiece to a vector. More...

virtual Eigen::VectorXd const &	ApplyJacobian (unsigned int const outputDimWrt, unsigned int const inputDimWrt, ref_vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &vec)

Eigen::VectorXd const &	ApplyJacobian (unsigned int outWrt, unsigned int inWrt, Eigen::VectorXd const &last, Eigen::VectorXd const &sens)

template<typename... Args>
Eigen::VectorXd const &	ApplyJacobian (unsigned int wrtOut, unsigned int wrtIn, Args const &... args)

virtual Eigen::VectorXd	GradientByFD (unsigned int const outputDimWrt, unsigned int const inputDimWrt, std::vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sensitivity)

virtual Eigen::VectorXd	GradientByFD (unsigned int const outputDimWrt, unsigned int const inputDimWrt, ref_vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sensitivity)

virtual Eigen::MatrixXd	JacobianByFD (unsigned int const outputDimWrt, unsigned int const inputDimWrt, std::vector< Eigen::VectorXd > const &input)

virtual Eigen::MatrixXd	JacobianByFD (unsigned int const outputDimWrt, unsigned int const inputDimWrt, ref_vector< Eigen::VectorXd > const &input)

virtual Eigen::VectorXd	ApplyJacobianByFD (unsigned int const outputDimWrt, unsigned int const inputDimWrt, std::vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &vec)

virtual Eigen::VectorXd	ApplyJacobianByFD (unsigned int const outputDimWrt, unsigned int const inputDimWrt, ref_vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &vec)

virtual Eigen::VectorXd const &	ApplyHessian (unsigned int const outWrt, unsigned int const inWrt1, unsigned int const inWrt2, std::vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sens, Eigen::VectorXd const &vec)

virtual Eigen::VectorXd const &	ApplyHessian (unsigned int const outWrt, unsigned int const inWrt1, unsigned int const inWrt2, ref_vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sens, Eigen::VectorXd const &vec)

virtual Eigen::VectorXd	ApplyHessianByFD (unsigned int const outWrt, unsigned int const inWrt1, unsigned int const inWrt2, std::vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sens, Eigen::VectorXd const &vec)

virtual Eigen::VectorXd	ApplyHessianByFD (unsigned int const outWrt, unsigned int const inWrt1, unsigned int const inWrt2, ref_vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sens, Eigen::VectorXd const &vec)

void	EnableCache ()

void	DisableCache ()

bool	CacheStatus () const

virtual void	SetWarnLevel (unsigned int newLevel)

Public Member Functions inherited from muq::Modeling::WorkPiece
	WorkPiece ()
	Create a muq::Modeling::WorkPiece with no fixed number of inputs and outputs and variable input/output types. More...

	WorkPiece (int const num, WorkPiece::Fix const fix=WorkPiece::Fix::Inputs)
	Create a muq::Modeling::WorkPiece with either a fixed number of inputs or outputs and variable input/output types. More...

	WorkPiece (int const numIns, int const numOuts)
	Create a muq::Modeling::WorkPiece with a fixed number of inputs and outputs but variable input/output types. More...

	WorkPiece (std::vector< std::string > const &types, WorkPiece::Fix const fix=WorkPiece::Fix::Inputs)
	Create a muq::Modeling::WorkPiece with either a fixed number of inputs with specified types or a fixed number of outputs with specified types. More...

	WorkPiece (std::map< unsigned int, std::string > const &types, WorkPiece::Fix const fix=WorkPiece::Fix::Inputs)
	Create a muq::Modeling::WorkPiece where either some of the inputs have specified types or some of the outputs have specified types. More...

	WorkPiece (std::map< unsigned int, std::string > const &types, int const num, WorkPiece::Fix const fixTypes=WorkPiece::Fix::Inputs, WorkPiece::Fix const fixNum=WorkPiece::Fix::Inputs)
	Create a muq::Modeling::WorkPiece where either some of the inputs have specified types or some of the outputs have specified types and either the number of inputs or the number of outputs is fixed. More...

	WorkPiece (std::vector< std::string > const &types, int const num)
	Create a muq::Modeling::WorkPiece with a fixed number of inputs with specified types and a fixed number of outputs (of uknown type) More...

	WorkPiece (int const num, std::vector< std::string > const &types)
	Create a muq::Modeling::WorkPiece with a fixed number of outputs with specified types and a fixed number of inputs (of uknown type) More...

	WorkPiece (std::map< unsigned int, std::string > const &inTypes, int const numIns, int const numOuts)
	Create a muq::Modeling::WorkPiece where some of the inputs are known and we know the input and output numbers. More...

	WorkPiece (int const numIns, std::map< unsigned int, std::string > const &outTypes, int const numOuts)
	Create a muq::Modeling::WorkPiece where some of the outputs are known and we know the input and output numbers. More...

	WorkPiece (std::vector< std::string > const &inTypes, std::vector< std::string > const &outTypes)
	Create a muq::Modeling::WorkPiece with a fixed number of inputs and outputs with specified types. More...

	WorkPiece (std::map< unsigned int, std::string > const &inTypes, std::vector< std::string > const &outTypes)
	Create a muq::Modeling::WorkPiece where some of the inputs are known and all of the outputs have specified types. More...

	WorkPiece (std::map< unsigned int, std::string > const &inTypes, int const num, std::vector< std::string > const &outTypes)
	Create a muq::Modeling::WorkPiece where some of the inputs are known with a known number of inputs and all of the outputs have specified types. More...

	WorkPiece (std::vector< std::string > const &inTypes, std::map< unsigned int, std::string > const &outTypes)
	Create a muq::Modeling::WorkPiece where some of the outputs and all of the inputs have specified types. More...

	WorkPiece (std::vector< std::string > const &inTypes, std::map< unsigned int, std::string > const &outTypes, int const num)
	Create a muq::Modeling::WorkPiece where some of the outputs with a known number of outputs and all of the inputs have specified types. More...

	WorkPiece (std::map< unsigned int, std::string > const &inTypes, std::map< unsigned int, std::string > const &outTypes)
	Create a muq::Mdoeling::WorkPiece where some of the inputs and some of the outputs have specified types. More...

	WorkPiece (std::map< unsigned int, std::string > const &inTypes, int const numIn, std::map< unsigned int, std::string > const &outTypes)
	Create a muq::Mdoeling::WorkPiece where some of the inputs and some of the outputs have specified types with a fixed number of inputs. More...

	WorkPiece (std::map< unsigned int, std::string > const &inTypes, std::map< unsigned int, std::string > const &outTypes, int const numOut)
	Create a muq::Mdoeling::WorkPiece where some of the inputs and some of the outputs have specified types with a fixed number of outputs. More...

	WorkPiece (std::map< unsigned int, std::string > const &inTypes, int const numIn, std::map< unsigned int, std::string > const &outTypes, int const numOut)
	Create a muq::Mdoeling::WorkPiece where some of the inputs and some of the outputs have specified types with a fixed number of inputs and outputs. More...

virtual	~WorkPiece ()
	Default destructor. More...

std::vector< boost::any > const &	Evaluate (std::vector< boost::any > const &ins)
	Evaluate this muq::Modeling::WorkPiece. More...

std::vector< boost::any > const &	Evaluate (ref_vector< boost::any > const &ins)
	Evaluate this muq::Modeling::WorkPiece using references to the inputs. More...

std::vector< boost::any > const &	Evaluate ()
	Evaluate this muq::Modeling::WorkPiece in the case that there are no inputs. More...

template<typename... Args>
std::vector< boost::any > const &	Evaluate (Args... args)
	Evalaute this muq::Modeling::WorkPiece using multiple arguments. More...

std::string const &	Name ()
	Get the (unique) name of this work piece. More...

void	SetName (std::string const &newName)
	Set the name of this work piece. More...

std::string	InputType (unsigned int inputNum, bool const demangle=true) const
	Get the input type (if we know it) for a specific input. More...

int	InputSize (unsigned int inputNum) const
	Get the length of a vector valued input with fixed size. More...

void	SetInputSize (unsigned int inputNum, int newSize)

std::string	OutputType (unsigned int outputNum, bool const demangle=true) const
	Get the output type (if we know it) for a specific output. More...

std::map< unsigned int, std::string >	OutputTypes () const
	Get the output types. More...

std::map< unsigned int, std::string >	InputTypes () const
	Get the input types. More...

unsigned int	ID () const
	Get the unique ID number. More...

Additional Inherited Members
Static Public Member Functions inherited from muq::Modeling::WorkPiece
static ref_vector< const boost::any >	ToRefVector (std::vector< boost::any > const &anyVec)
	Create vector of references from a vector of boost::any's. More...

static ref_vector< const Eigen::VectorXd >	ToRefVector (std::vector< Eigen::VectorXd > const &anyVec)

Public Attributes inherited from muq::Modeling::ModPiece
const Eigen::VectorXi	inputSizes

const Eigen::VectorXi	outputSizes

Public Attributes inherited from muq::Modeling::WorkPiece
int	numInputs
	The number of inputs. More...

int	numOutputs
	The number of outputs. More...

Constructor & Destructor Documentation

◆ CostFunction()

muq::Optimization::CostFunction::CostFunction ( unsigned int dim )

inline

Parameters

[in] dim The number of the decision variables in \(x\).

Definition at line 80 of file CostFunction.h.

◆ ~CostFunction()

virtual muq::Optimization::CostFunction::~CostFunction ( )

virtualdefault

Member Function Documentation

◆ ApplyHessian() [1/2]

virtual Eigen::VectorXd muq::Optimization::CostFunction::ApplyHessian	(	Eigen::VectorXd const &	evalPt,
		Eigen::VectorXd const &	vec
	)

inlinevirtual

The Hessian of the cost function.

Parameters

[in]	inputDimWrt	Which input are we taking the 2nd derivative with respect to?
[in]	input	The inputs \(x\), \(\theta_{1:n}\)
[in]	vec	Vector to which the Hessian is applied

Returns: The Hessian action on vec

Definition at line 124 of file CostFunction.h.

References SetPoint().

◆ ApplyHessian() [2/2]

Eigen::VectorXd CostFunction::ApplyHessian ( Eigen::VectorXd const & vec )

virtual

Reimplemented in muq::Optimization::ModPieceCostFunction.

Definition at line 67 of file CostFunction.cpp.

References muq::Modeling::ModPiece::ApplyHessianByFD().

◆ ApplyHessianImpl()

void CostFunction::ApplyHessianImpl	(	unsigned int	outWrt,
		unsigned int	inWrt1,
		unsigned int	inWrt2,
		muq::Modeling::ref_vector< Eigen::VectorXd > const &	input,
		Eigen::VectorXd const &	sensitivity,
		Eigen::VectorXd const &	vec
	)

overrideprivatevirtual

Reimplemented from muq::Modeling::ModPiece.

Definition at line 79 of file CostFunction.cpp.

◆ Cost() [1/2]

virtual double muq::Optimization::CostFunction::Cost ( )

pure virtual

Implemented in muq::Optimization::ModPieceCostFunction, and muq::Approximation::Regression::PoisednessCost.

◆ Cost() [2/2]

virtual double muq::Optimization::CostFunction::Cost ( Eigen::VectorXd const & x )

inlinevirtual

The value of the cost function.

Parameters

[in] input The inputs \(x\), \(\theta_{1:n}\)

Returns: The value of the cost function

Definition at line 92 of file CostFunction.h.

References Cost(), SetPoint(), and x.

Referenced by Cost().

◆ EvaluateImpl()

void CostFunction::EvaluateImpl ( muq::Modeling::ref_vector< Eigen::VectorXd > const & input )

overrideprivatevirtual

The value of the cost function.

Parameters

[in] args The inputs \(x\), \(\theta_{1:n}\)

Returns: The value of the cost function

Implements muq::Modeling::ModPiece.

Definition at line 7 of file CostFunction.cpp.

◆ Gradient() [1/2]

Eigen::VectorXd CostFunction::Gradient ( )

virtual

Reimplemented in muq::Optimization::ModPieceCostFunction.

Definition at line 24 of file CostFunction.cpp.

References muq::Modeling::ModPiece::GradientByFD().

◆ Gradient() [2/2]

virtual Eigen::VectorXd muq::Optimization::CostFunction::Gradient ( Eigen::VectorXd const & evalPt )

inlinevirtual

Definition at line 95 of file CostFunction.h.

References Gradient(), and SetPoint().

Referenced by Gradient().

◆ GradientImpl()

void CostFunction::GradientImpl	(	unsigned int	outputDimWrt,
		unsigned int	inputDimWrt,
		muq::Modeling::ref_vector< Eigen::VectorXd > const &	input,
		Eigen::VectorXd const &	sensitivity
	)

overrideprivatevirtual

Compute the gradient of the cost function.

Parameters

[in]	inputDimWrt	Which input are we taking the derivative with respect to?
[in]	args	The inputs \(x\), \(\theta_{1:n}\) and the sensitivity vector

Reimplemented from muq::Modeling::ModPiece.

Reimplemented in muq::Approximation::Regression::PoisednessCost.

Definition at line 12 of file CostFunction.cpp.

◆ Hessian() [1/2]

Eigen::MatrixXd CostFunction::Hessian ( )

virtual

Definition at line 30 of file CostFunction.cpp.

◆ Hessian() [2/2]

virtual Eigen::MatrixXd muq::Optimization::CostFunction::Hessian ( Eigen::VectorXd const & evalPt )

inlinevirtual

The Hessian of the cost function.

Parameters

[in]	inputDimWrt	Which input are we taking the 2nd derivative with respect to?
[in]	input	The inputs \(x\), \(\theta_{1:n}\)

Returns: The Hessian of the cost function

Definition at line 105 of file CostFunction.h.

References Hessian(), and SetPoint().

Referenced by Hessian().

◆ HessianByFD() [1/2]

Eigen::MatrixXd CostFunction::HessianByFD ( )

virtual

Definition at line 35 of file CostFunction.cpp.

◆ HessianByFD() [2/2]

virtual Eigen::MatrixXd muq::Optimization::CostFunction::HessianByFD ( Eigen::VectorXd const & evalPt )

inlinevirtual

The Hessian of the cost function using finite difference.

Parameters

[in]	inputDimWrt	Which input are we taking the 2nd derivative with respect to?
[in]	input	The inputs \(x\), \(\theta_{1:n}\)

Returns: The Hessian of the cost function

Definition at line 114 of file CostFunction.h.

References HessianByFD(), and SetPoint().

Referenced by HessianByFD().

◆ JacobianImpl()

void CostFunction::JacobianImpl	(	unsigned int	outputDimWrt,
		unsigned int	inputDimWrt,
		muq::Modeling::ref_vector< Eigen::VectorXd > const &	input
	)

overrideprivatevirtual

Reimplemented from muq::Modeling::ModPiece.

Definition at line 72 of file CostFunction.cpp.

◆ SetPoint()

void CostFunction::SetPoint ( Eigen::VectorXd const & evalPt )

virtual

Definition at line 19 of file CostFunction.cpp.

Referenced by ApplyHessian(), Cost(), Gradient(), Hessian(), and HessianByFD().

Member Data Documentation

◆ x

Eigen::VectorXd muq::Optimization::CostFunction::x

protected

Definition at line 130 of file CostFunction.h.

Referenced by muq::Optimization::ModPieceCostFunction::ApplyHessian(), muq::Optimization::ModPieceCostFunction::Cost(), Cost(), and muq::Optimization::ModPieceCostFunction::Gradient().

The documentation for this class was generated from the following files:

doxygen_prep/MUQ/Optimization/CostFunction.h
doxygen_prep/modules/Optimization/src/CostFunction.cpp

Detailed Description

Public Member Functions

Additional Inherited Members

Constructor & Destructor Documentation

◆ CostFunction()

◆ ~CostFunction()

Member Function Documentation

◆ ApplyHessian() [1/2]

◆ ApplyHessian() [2/2]

◆ ApplyHessianImpl()

◆ Cost() [1/2]

◆ Cost() [2/2]

◆ EvaluateImpl()

◆ Gradient() [1/2]

◆ Gradient() [2/2]

◆ GradientImpl()

◆ Hessian() [1/2]

◆ Hessian() [2/2]

◆ HessianByFD() [1/2]

◆ HessianByFD() [2/2]

◆ JacobianImpl()

◆ SetPoint()

Member Data Documentation

◆ x