MUQ  0.4.3
muq::Modeling::MultiLogisticLikelihood Class Reference

Class that defines the likelihood function for multinomial logistic regression. More...

#include <MultiLogisticLikelihood.h>

Inheritance diagram for muq::Modeling::MultiLogisticLikelihood:

Detailed Description

Class that defines the likelihood function for multinomial logistic regression.

Consider a discrete random variable \(y\in\{1,2,\ldots, M\}\) that can take one of \(M\) values. Assume this random depends on another indpendent variable \(x\in\mathbb{R}^N\) and we are interested in modeling the probabilities \(p_i(x) = \mathbb{P}[y=i]\), which are functions of the independent varible \(x\). Logistic regression is the classic approach when \(M=2\) and Multinomial Logistic Regression is a natural way to extend logistic regression to the case where \(M>2\).

The basic idea is to model the log probabilities \(\log(p_i)\) through an expression of the form

\[ \log(p_i) = f_i(x) - \log(z(x)), \]

where \(\log(z(x))\) is a normalization term that ensure the sum of the probabilities is equal to \(1\). More precisely,

\[ z = \sum_{i=1}^M \exp(f_i(x)). \]

Typically, the functions \(f_i\) would also be represent through an expansion of the form

\[ \log(p_i) = -\log(z) + \sum_{j=1}^P \theta_j \phi_j(x), \]

for some basis functions \(\phi_j\).

This class evaluates \(\log(p_i)\) for each \(i\) given the values of \(f_i(x)\) and then computes the likelihood of \(K\) observations \(y_1,\ldots, y_K\). The data is passed to the constructor.

This piece assumes that each observation \(y_k\) corresponds to a condition \(x_k\). The input to this function is thus an unrolled vector containing \(f_i(x_k)\) for all \(i\in\left{1,\ldots,M\right}\) and \(k\in\{1,\ldots, K\}\). It is assumed that \(i\) is the faster index, so that the input vector would look like \([f_1(x_1),\ldots, f_M(x_1),f_1(x_2),\ldots]\).

This piece returns a scalar value, with the value of the loglikelihood

\[ \log \pi(y| x) = \sum_{k=1}^K \log(p_{y_k}(x_k)) \]

Definition at line 51 of file MultiLogisticLikelihood.h.

Public Member Functions

 MultiLogisticLikelihood (unsigned int numClasses, Eigen::VectorXi const &data)
 
- Public Member Functions inherited from muq::Modeling::ModPiece
 ModPiece (std::vector< int > const &inputSizes, std::vector< int > const &outputSizes)
 
 ModPiece (Eigen::VectorXi const &inputSizes, Eigen::VectorXi const &outputSizes)
 
virtual ~ModPiece ()=default
 
virtual double GetRunTime (const std::string &method="Evaluate") const override
 Get the average run time for one of the implemented methods. More...
 
virtual void ResetCallTime () override
 Resets the number of call and times. More...
 
virtual unsigned long int GetNumCalls (const std::string &method="Evaluate") const override
 get the number of times one of the implemented methods has been called. More...
 
virtual std::vector< Eigen::VectorXd > const & Evaluate (std::vector< Eigen::VectorXd > const &input)
 Evaluate the ModPiece. More...
 
virtual std::vector< Eigen::VectorXd > const & Evaluate (ref_vector< Eigen::VectorXd > const &input)
 
 VARIADIC_TO_REFVECTOR (Evaluate, Eigen::VectorXd, std::vector< Eigen::VectorXd > const &)
 
virtual Eigen::VectorXd const & Gradient (unsigned int const outputDimWrt, unsigned int const inputDimWrt, std::vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sensitivity)
 Compute the Gradient \(J^Tv\). More...
 
virtual Eigen::VectorXd const & Gradient (unsigned int const outputDimWrt, unsigned int const inputDimWrt, ref_vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sensitivity)
 
Eigen::VectorXd const & Gradient (unsigned int outWrt, unsigned int inWrt, Eigen::VectorXd const &last, Eigen::VectorXd const &sens)
 
template<typename... Args>
Eigen::VectorXd const & Gradient (unsigned int wrtOut, unsigned int wrtIn, Args const &... args)
 
Eigen::VectorXd const & ApplyHessian (unsigned int outWrt, unsigned int inWrt1, unsigned int inWrt2, Eigen::VectorXd const &last, Eigen::VectorXd const &sens, Eigen::VectorXd const &vec)
 
template<typename... Args>
Eigen::VectorXd const & ApplyHessian (unsigned int wrtOut, unsigned int wrtIn1, unsigned int wrtIn2, Args const &... args)
 
virtual Eigen::MatrixXd const & Jacobian (unsigned int const outputDimWrt, unsigned int const inputDimWrt, std::vector< Eigen::VectorXd > const &input)
 Compute the Jacobian of this ModPiece. More...
 
virtual Eigen::MatrixXd const & Jacobian (unsigned int const outputDimWrt, unsigned int const inputDimWrt, ref_vector< Eigen::VectorXd > const &input)
 
template<typename... Args>
Eigen::MatrixXd const & Jacobian (unsigned int outWrt, unsigned int inWrt, Args const &... args)
 
template<typename... Args>
Eigen::MatrixXd JacobianByFD (unsigned int outWrt, unsigned int inWrt, Args const &... args)
 
template<typename... Args>
Eigen::MatrixXd ApplyJacobianByFD (unsigned int outWrt, unsigned int inWrt, Args const &... args)
 
virtual Eigen::VectorXd const & ApplyJacobian (unsigned int const outputDimWrt, unsigned int const inputDimWrt, std::vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &vec)
 Apply the Jacobian of this ModPiece to a vector. More...
 
virtual Eigen::VectorXd const & ApplyJacobian (unsigned int const outputDimWrt, unsigned int const inputDimWrt, ref_vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &vec)
 
Eigen::VectorXd const & ApplyJacobian (unsigned int outWrt, unsigned int inWrt, Eigen::VectorXd const &last, Eigen::VectorXd const &sens)
 
template<typename... Args>
Eigen::VectorXd const & ApplyJacobian (unsigned int wrtOut, unsigned int wrtIn, Args const &... args)
 
virtual Eigen::VectorXd GradientByFD (unsigned int const outputDimWrt, unsigned int const inputDimWrt, std::vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sensitivity)
 
virtual Eigen::VectorXd GradientByFD (unsigned int const outputDimWrt, unsigned int const inputDimWrt, ref_vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sensitivity)
 
virtual Eigen::MatrixXd JacobianByFD (unsigned int const outputDimWrt, unsigned int const inputDimWrt, std::vector< Eigen::VectorXd > const &input)
 
virtual Eigen::MatrixXd JacobianByFD (unsigned int const outputDimWrt, unsigned int const inputDimWrt, ref_vector< Eigen::VectorXd > const &input)
 
virtual Eigen::VectorXd ApplyJacobianByFD (unsigned int const outputDimWrt, unsigned int const inputDimWrt, std::vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &vec)
 
virtual Eigen::VectorXd ApplyJacobianByFD (unsigned int const outputDimWrt, unsigned int const inputDimWrt, ref_vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &vec)
 
virtual Eigen::VectorXd const & ApplyHessian (unsigned int const outWrt, unsigned int const inWrt1, unsigned int const inWrt2, std::vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sens, Eigen::VectorXd const &vec)
 
virtual Eigen::VectorXd const & ApplyHessian (unsigned int const outWrt, unsigned int const inWrt1, unsigned int const inWrt2, ref_vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sens, Eigen::VectorXd const &vec)
 
virtual Eigen::VectorXd ApplyHessianByFD (unsigned int const outWrt, unsigned int const inWrt1, unsigned int const inWrt2, std::vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sens, Eigen::VectorXd const &vec)
 
virtual Eigen::VectorXd ApplyHessianByFD (unsigned int const outWrt, unsigned int const inWrt1, unsigned int const inWrt2, ref_vector< Eigen::VectorXd > const &input, Eigen::VectorXd const &sens, Eigen::VectorXd const &vec)
 
void EnableCache ()
 
void DisableCache ()
 
bool CacheStatus () const
 
virtual void SetWarnLevel (unsigned int newLevel)
 
- Public Member Functions inherited from muq::Modeling::WorkPiece
 WorkPiece ()
 Create a muq::Modeling::WorkPiece with no fixed number of inputs and outputs and variable input/output types. More...
 
 WorkPiece (int const num, WorkPiece::Fix const fix=WorkPiece::Fix::Inputs)
 Create a muq::Modeling::WorkPiece with either a fixed number of inputs or outputs and variable input/output types. More...
 
 WorkPiece (int const numIns, int const numOuts)
 Create a muq::Modeling::WorkPiece with a fixed number of inputs and outputs but variable input/output types. More...
 
 WorkPiece (std::vector< std::string > const &types, WorkPiece::Fix const fix=WorkPiece::Fix::Inputs)
 Create a muq::Modeling::WorkPiece with either a fixed number of inputs with specified types or a fixed number of outputs with specified types. More...
 
 WorkPiece (std::map< unsigned int, std::string > const &types, WorkPiece::Fix const fix=WorkPiece::Fix::Inputs)
 Create a muq::Modeling::WorkPiece where either some of the inputs have specified types or some of the outputs have specified types. More...
 
 WorkPiece (std::map< unsigned int, std::string > const &types, int const num, WorkPiece::Fix const fixTypes=WorkPiece::Fix::Inputs, WorkPiece::Fix const fixNum=WorkPiece::Fix::Inputs)
 Create a muq::Modeling::WorkPiece where either some of the inputs have specified types or some of the outputs have specified types and either the number of inputs or the number of outputs is fixed. More...
 
 WorkPiece (std::vector< std::string > const &types, int const num)
 Create a muq::Modeling::WorkPiece with a fixed number of inputs with specified types and a fixed number of outputs (of uknown type) More...
 
 WorkPiece (int const num, std::vector< std::string > const &types)
 Create a muq::Modeling::WorkPiece with a fixed number of outputs with specified types and a fixed number of inputs (of uknown type) More...
 
 WorkPiece (std::map< unsigned int, std::string > const &inTypes, int const numIns, int const numOuts)
 Create a muq::Modeling::WorkPiece where some of the inputs are known and we know the input and output numbers. More...
 
 WorkPiece (int const numIns, std::map< unsigned int, std::string > const &outTypes, int const numOuts)
 Create a muq::Modeling::WorkPiece where some of the outputs are known and we know the input and output numbers. More...
 
 WorkPiece (std::vector< std::string > const &inTypes, std::vector< std::string > const &outTypes)
 Create a muq::Modeling::WorkPiece with a fixed number of inputs and outputs with specified types. More...
 
 WorkPiece (std::map< unsigned int, std::string > const &inTypes, std::vector< std::string > const &outTypes)
 Create a muq::Modeling::WorkPiece where some of the inputs are known and all of the outputs have specified types. More...
 
 WorkPiece (std::map< unsigned int, std::string > const &inTypes, int const num, std::vector< std::string > const &outTypes)
 Create a muq::Modeling::WorkPiece where some of the inputs are known with a known number of inputs and all of the outputs have specified types. More...
 
 WorkPiece (std::vector< std::string > const &inTypes, std::map< unsigned int, std::string > const &outTypes)
 Create a muq::Modeling::WorkPiece where some of the outputs and all of the inputs have specified types. More...
 
 WorkPiece (std::vector< std::string > const &inTypes, std::map< unsigned int, std::string > const &outTypes, int const num)
 Create a muq::Modeling::WorkPiece where some of the outputs with a known number of outputs and all of the inputs have specified types. More...
 
 WorkPiece (std::map< unsigned int, std::string > const &inTypes, std::map< unsigned int, std::string > const &outTypes)
 Create a muq::Mdoeling::WorkPiece where some of the inputs and some of the outputs have specified types. More...
 
 WorkPiece (std::map< unsigned int, std::string > const &inTypes, int const numIn, std::map< unsigned int, std::string > const &outTypes)
 Create a muq::Mdoeling::WorkPiece where some of the inputs and some of the outputs have specified types with a fixed number of inputs. More...
 
 WorkPiece (std::map< unsigned int, std::string > const &inTypes, std::map< unsigned int, std::string > const &outTypes, int const numOut)
 Create a muq::Mdoeling::WorkPiece where some of the inputs and some of the outputs have specified types with a fixed number of outputs. More...
 
 WorkPiece (std::map< unsigned int, std::string > const &inTypes, int const numIn, std::map< unsigned int, std::string > const &outTypes, int const numOut)
 Create a muq::Mdoeling::WorkPiece where some of the inputs and some of the outputs have specified types with a fixed number of inputs and outputs. More...
 
virtual ~WorkPiece ()
 Default destructor. More...
 
std::vector< boost::any > const & Evaluate (std::vector< boost::any > const &ins)
 Evaluate this muq::Modeling::WorkPiece. More...
 
std::vector< boost::any > const & Evaluate (ref_vector< boost::any > const &ins)
 Evaluate this muq::Modeling::WorkPiece using references to the inputs. More...
 
std::vector< boost::any > const & Evaluate ()
 Evaluate this muq::Modeling::WorkPiece in the case that there are no inputs. More...
 
template<typename... Args>
std::vector< boost::any > const & Evaluate (Args... args)
 Evalaute this muq::Modeling::WorkPiece using multiple arguments. More...
 
std::string const & Name ()
 Get the (unique) name of this work piece. More...
 
void SetName (std::string const &newName)
 Set the name of this work piece. More...
 
std::string InputType (unsigned int inputNum, bool const demangle=true) const
 Get the input type (if we know it) for a specific input. More...
 
int InputSize (unsigned int inputNum) const
 Get the length of a vector valued input with fixed size. More...
 
void SetInputSize (unsigned int inputNum, int newSize)
 
std::string OutputType (unsigned int outputNum, bool const demangle=true) const
 Get the output type (if we know it) for a specific output. More...
 
std::map< unsigned int, std::string > OutputTypes () const
 Get the output types. More...
 
std::map< unsigned int, std::string > InputTypes () const
 Get the input types. More...
 
unsigned int ID () const
 Get the unique ID number. More...
 

Additional Inherited Members

- Static Public Member Functions inherited from muq::Modeling::WorkPiece
static ref_vector< const boost::any > ToRefVector (std::vector< boost::any > const &anyVec)
 Create vector of references from a vector of boost::any's. More...
 
static ref_vector< const Eigen::VectorXd > ToRefVector (std::vector< Eigen::VectorXd > const &anyVec)
 
- Public Attributes inherited from muq::Modeling::ModPiece
const Eigen::VectorXi inputSizes
 
const Eigen::VectorXi outputSizes
 
- Public Attributes inherited from muq::Modeling::WorkPiece
int numInputs
 The number of inputs. More...
 
int numOutputs
 The number of outputs. More...
 

Constructor & Destructor Documentation

◆ MultiLogisticLikelihood()

MultiLogisticLikelihood::MultiLogisticLikelihood ( unsigned int  numClasses,
Eigen::VectorXi const &  data 
)

Definition at line 5 of file MultiLogisticLikelihood.cpp.

Member Function Documentation

◆ ApplyJacobianImpl()

void MultiLogisticLikelihood::ApplyJacobianImpl ( unsigned int const  outputDimWrt,
unsigned int const  inputDimWrt,
ref_vector< Eigen::VectorXd > const &  input,
Eigen::VectorXd const &  vec 
)
overrideprivatevirtual

◆ EvaluateImpl()

void MultiLogisticLikelihood::EvaluateImpl ( muq::Modeling::ref_vector< Eigen::VectorXd > const &  inputs)
overrideprivatevirtual

◆ GradientImpl()

void MultiLogisticLikelihood::GradientImpl ( unsigned int const  outputDimWrt,
unsigned int const  inputDimWrt,
ref_vector< Eigen::VectorXd > const &  input,
Eigen::VectorXd const &  sensitivity 
)
overrideprivatevirtual

◆ JacobianImpl()

void MultiLogisticLikelihood::JacobianImpl ( unsigned int const  outputDimWrt,
unsigned int const  inputDimWrt,
ref_vector< Eigen::VectorXd > const &  input 
)
overrideprivatevirtual

Member Data Documentation

◆ data

Eigen::VectorXi muq::Modeling::MultiLogisticLikelihood::data
private

Definition at line 75 of file MultiLogisticLikelihood.h.

Referenced by EvaluateImpl(), and GradientImpl().

◆ numClasses

const unsigned int muq::Modeling::MultiLogisticLikelihood::numClasses
private

Definition at line 74 of file MultiLogisticLikelihood.h.

Referenced by EvaluateImpl(), and GradientImpl().


The documentation for this class was generated from the following files: