NeuralEngine A Game Engine with embeded Machine Learning algorithms based on Gaussian Processes. |

Public Member Functions | Protected Member Functions | Private Member Functions | Private Attributes | Friends | List of all members
NeuralEngine::MachineLearning::GPModels::AEP::SGPLVM< Scalar > Class Template Reference

Sparse GPLVM via Approximated Expectation Propagation (AEP). More...

`#include <FgAEPSparseGPLVM.h>`

Inheritance diagram for NeuralEngine::MachineLearning::GPModels::AEP::SGPLVM< Scalar >:
[legend]
Collaboration diagram for NeuralEngine::MachineLearning::GPModels::AEP::SGPLVM< Scalar >:
[legend]

## Public Member Functions
| SGPLVM (const af::array &Y, int latentDimension, int numInducing=200, Scalar alpha=1.0, Scalar priorMean=0.0, Scalar priorVariance=1.0, PropagationMode probMode=PropagationMode::MomentMatching, LogLikType lType=LogLikType::Gaussian, XInit emethod=XInit::pca)
| Constructor. More...
| SGPLVM ()
| Default constructor. More...
virtual Scalar | Function (const af::array &x, af::array &outGradient) override
| Cost function the given parameter inputs. More...
Public Member Functions inherited from NeuralEngine::MachineLearning::GPModels::SparseGPLVMBaseModel< Scalar >
| SparseGPLVMBaseModel (const af::array &Y, int latentDimension, Scalar priorMean=0.0, Scalar priorVariance=1.0, int numInducing=200, LogLikType lType=LogLikType::Gaussian, XInit emethod=XInit::pca)
| Constructor. More...
| SparseGPLVMBaseModel ()
| Default constructor. More...
virtual | ~SparseGPLVMBaseModel ()
| Destructor. More...
virtual void | PredictF (const af::array &testInputs, af::array &mf, af::array &vf) override
| Predict noise free functions values \(\mathbf{F}_*\). More...
virtual void | SampleY (const af::array inputs, int numSamples, af::array &outFunctions) override
| Generate function samples from posterior. More...
virtual bool | Init () override
| Initializes the model. More...
virtual int | GetNumParameters () override
| Gets number of parameters. More...
virtual void | SetParameters (const af::array &param) override
| Sets the parameters for each optimization iteration. More...
virtual af::array | GetParameters () override
| Gets the parameters for each optimization iteration. More...
virtual void | UpdateParameters () override
| Updates the parameters. More...
virtual void | FixKernelParameters (bool isfixed) override
| Sets fixation for hyperparameters. More...
virtual void | FixInducing (bool isfixed) override
| Set fixation for inducing inputs. More...
Public Member Functions inherited from NeuralEngine::MachineLearning::GPModels::GPLVMBaseModel< Scalar >
| GPLVMBaseModel (const af::array &Y, int latentDimension, Scalar priorMean=0.0, Scalar priorVariance=1.0, LogLikType lType=LogLikType::Gaussian, XInit emethod=XInit::pca)
| Constructor. More...
| GPLVMBaseModel ()
| Default constructor. More...
virtual | ~GPLVMBaseModel ()
| Destructor. More...
virtual void | Optimise (OptimizerType method=L_BFGS, Scalar tol=0.0, bool reinit_hypers=true, int maxiter=1000, int mb_size=0, LineSearchType lsType=MoreThuente, bool disp=true, int *cycle=nullptr) override
| Optimizes the model parameters for best fit. More...
virtual bool | Init (af::array &mx)
| Initializes the model. More...
virtual bool | Init () override
| Initializes the model. More...
virtual void | PosteriorLatents (af::array &mx, af::array &vx)
| Get posterior distribution of latent variables /f$\mathbf{X}/f$. More...
virtual int | GetNumParameters () override
| Gets number of parameters. More...
virtual void | SetParameters (const af::array &param) override
| Sets the parameters for each optimization iteration. More...
virtual af::array | GetParameters () override
| Gets the parameters for each optimization iteration. More...
virtual void | UpdateParameters () override
| Updates the parameters. More...
virtual void | FixKernelParameters (bool isfixed)
| Sets fixation for hyperparameters. More...
virtual void | FixInducing (bool isfixed)
| Set fixation for inducing inputs. More...
void | FixLatents (bool isFixed)
af::array | GetMeanGradient ()
| Gets prior mean gradient. More...
af::array | GetVarGradient ()
| Gets prior variance gradient. More...
void | SetPrior (const af::array mean, const af::array var)
| Sets the prior. More...
void | SetPriorCavity (const af::array meanCav, const af::array varCav)
| Sets the cavity prior. More...
void | SetLatentGradient (const af::array &dmParent, const af::array &dvParent)
| Sets latent gradient. More...
void | SetLatentGradientCavity (const af::array &dmParent, const af::array &dvParent)
| Sets the latent cavity gradient. More...
int | GetLatentDimension ()
| Gets latent dimension. More...
void | SetBackConstraint (IBackconstraint< Scalar > *constraint)
| Sets a back-constraint. More...
IBackconstraint< Scalar > * | GetBackConstraint ()
| Gets the back-constraint. More...
void | SetStyles (std::map< std::string, Style< Scalar > > *styles)
| Sets the syles. More...
void | AddStyle (Style< Scalar > style)
| Adds a style. More...
std::map< std::string, Style< Scalar > > * | GetStyles ()
| Gets the styles. More...
Public Member Functions inherited from NeuralEngine::MachineLearning::GPModels::GPBaseModel< Scalar >
| GPBaseModel (const af::array &Y, LogLikType lType=LogLikType::Gaussian, ModelType mtype=ModelType::GPR)
| Constructor. More...
| GPBaseModel ()
| Default Constructor. More...
virtual | ~GPBaseModel ()
| Destructor. More...
virtual void | Optimise (OptimizerType method=L_BFGS, Scalar tol=0.0, bool reinit_hypers=true, int maxiter=1000, int mb_size=0, LineSearchType lsType=MoreThuente, bool disp=true, int *cycle=nullptr)
| Optimizes the model parameters for best fit. More...
virtual bool | Init ()
| Initializes the model. More...
virtual void | PredictF (const af::array &testInputs, af::array &mf, af::array &vf)
| Predict noise free functions values \(\mathbf{F}_*\). More...
virtual void | PredictY (const af::array &testInputs, af::array &my, af::array &vy)
| Prediction of test outputs \(\mathbf{Y}_*\). More...
virtual void | SampleY (const af::array inputs, int numSamples, af::array &outFunctions)
| Generate function samples from posterior. More...
virtual void | AddData (const af::array Ytrain)
| Adds training data to the model. More...
af::array | GetTrainingData ()
| Gets the training data set Y. More...
void | SetTrainingData (af::array &data)
| Sets training data Y. More...
virtual int | GetNumParameters ()
| Gets number of parameters. More...
virtual void | SetParameters (const af::array &param)
| Sets the parameters for each optimization iteration. More...
virtual af::array | GetParameters ()
| Gets the parameters for each optimization iteration. More...
virtual void | UpdateParameters ()
| Updates the parameters. More...
virtual void | FixLikelihoodParameters (bool isfixed)
| Sets the likelihood parameters to be fixed or not for optimization. More...
void | SetSegments (af::array segments)
| Sets fixation for hyperparameters. More...
af::array | GetSegments ()
| Gets the start index array for the sequences. More...
Public Member Functions inherited from NeuralEngine::MachineLearning::IModel< Scalar >
virtual Scalar | Function (const af::array &x, af::array &outGradient)
| Cost function the given x inputs. More...
virtual int | GetNumParameters ()=0
| Gets number of parameters to be optimized. More...
virtual void | SetParameters (const af::array &param)=0
| Sets the parameters for each optimization iteration. More...
virtual af::array | GetParameters ()=0
| Gets the parameters for each optimization iteration. More...
virtual void | UpdateParameters ()=0
| Updates the parameters. More...
int | GetDataLenght ()
| Gets data lenght. More...
int | GetDataDimensionality ()
| Gets data dimensionality. More...
ModelType | GetModelType ()
| Gets model type. More...
virtual void | SetBatchSize (int size)
| Sets batch size. More...
int | GetBatchSize ()
| Gets batch size. More...
void | SetIndexes (af::array &indexes)
| Sets the batch indexes. More...
Public Member Functions inherited from NeuralEngine::MachineLearning::GPNode< Scalar >
| GPNode ()
| Default constructor. More...
virtual | ~GPNode ()
| Destructor. More...
int | GetNumChildren () const
| Gets the number of children of this item. More...
int | AttachChild (std::shared_ptr< GPNode< Scalar > > const &child)
| Attaches a child. More...
int | DetachChild (std::shared_ptr< GPNode< Scalar > > const &child)
| Detaches a child. More...
std::shared_ptr< GPNode< Scalar > > | DetachChildAt (int i)
| Detaches a child at index. More...
void | DetachAllChildren ()
| Detach all children from this node. More...
std::shared_ptr< GPNode< Scalar > > | SetChild (int i, std::shared_ptr< GPNode< Scalar > > const &child)
| Sets a child. More...
std::shared_ptr< GPNode< Scalar > > | GetChild (int i)
| Gets a child at index. More...
GPNode< Scalar > * | GetParent ()
| Access to the parent object, which is null for the root of the hierarchy. More...
void | SetParent (GPNode< Scalar > *parent)
| Access to the parent object. Node calls this during attach/detach of children. More...

## Protected Member Functions
void | CavityLatents (af::array &mx, af::array &vx)
| Computes the cavity distribution of the latent points. More...
af::array | CavityGradientLatents (const af::array &dmx, const af::array &dvx, const af::array &m, const af::array &v)
| Gradient of the cavity distribution w.r.t. latent points. More...
Scalar | ComputePhiLatents (const af::array &mx, const af::array &vx, af::array *dmx=nullptr, af::array *dvx=nullptr)
| Calculates the likelihood contribution of the latent points and its gradients. More...
Protected Member Functions inherited from NeuralEngine::MachineLearning::GPModels::GPLVMBaseModel< Scalar >
virtual af::array | PosteriorGradientLatents (const af::array &dmx, const af::array &dvx)
| Posterior gradient of latent inputs /f$\mathbf{X}/f$. More...
virtual void | UpdateParametersInternal ()
| Updates the internal parameters. More...
virtual af::array | LatentGradient (const af::array &dm, const af::array &dv)
| Latent gradient. More...
Protected Member Functions inherited from NeuralEngine::MachineLearning::IModel< Scalar >
| IModel (int numData, int numDimension, ModelType type)
| Constructor. More...

## Private Member Functions
template<class Archive >
void | serialize (Archive &ar, unsigned int version)

## Private Attributes
Scalar | dAlpha
| fraction parameter More...
PropagationMode | pMode

## Friends
class | boost::serialization::access

## Additional Inherited Members
Protected Attributes inherited from NeuralEngine::MachineLearning::GPModels::SparseGPLVMBaseModel< Scalar >
int | ik
| number of inducing inputs More...
SparseGPBaseLayer< Scalar > * | gpLayer
| sparse Gaussian Process layer More...
Protected Attributes inherited from NeuralEngine::MachineLearning::GPModels::GPLVMBaseModel< Scalar >
int | iq
| latent dimension More...
Scalar | dPriorMean
| prior mean More...
Scalar | dPriorVariance
| prior variance More...
Scalar | dPriorX1
| prior /f$x_1/f$ More...
Scalar | dPriorX2
| prior /f$x_2/f$ More...
af::array | afFactorX1
| natural parameter factor 1 for latent variable More...
af::array | afFactorX2
| natural parameter factor 2 for latent variable More...
af::array | afPosteriorX1
| posterior natural parameter 1 for latent variable More...
af::array | afPosteriorX2
| posterior natural parameter 2 for latent variable More...
af::array | afPriorMean
| prior mean for hiersrchy mode More...
af::array | afPriorVariance
| prior variance for hiersrchy mode More...
af::array | afPriorMeanCav
| prior mean for hiersrchy mode More...
af::array | afPriorVarianceCav
| prior variance for hiersrchy mode More...
af::array | afPriorX1
| prior /f$x_1/f$ More...
af::array | afPriorX2
| prior /f$x_2/f$ More...
af::array | afPriorX1Cav
| prior /f$x_1/f$ More...
af::array | afPriorX2Cav
| prior /f$x_2/f$ More...
af::array | afGradMean
| prior mean gradient for hiersrchy mode More...
af::array | afGradVariance
| prior variance gradient for hiersrchy mode More...
af::array | afGradMeanCav
| prior mean gradient for hiersrchy mode More...
af::array | afGradVarianceCav
| prior variance gradient for hiersrchy mode More...
af::array | afLatentGradientX
| top down gradient More...
XInit | eEmMethod
IBackconstraint< Scalar > * | backConst
| back-constraint More...
std::map< std::string, Style< Scalar > > * | mStyles
| style variable More...
bool | bIsLatetsFixed
Protected Attributes inherited from NeuralEngine::MachineLearning::GPModels::GPBaseModel< Scalar >
bool | bInit
| check if model is initialized More...
af::array | afY
| training dataset, mean substracted More...
af::array | afBias
| the bias More...
af::array | afSegments
| Index of starting positions for all trials. More...
LikelihoodBaseLayer< Scalar > * | likLayer
| liklihood layer More...
Protected Attributes inherited from NeuralEngine::MachineLearning::IModel< Scalar >
ModelType | mType
int | iN
| dataset length More...
int | iD
| dataset dimension More...
int | iBatchSize
| size of the batch More...
af::array | afIndexes
| indexes of /f$\mathbf{X}/f$ for batch learning More...
af::dtype | m_dType
| floating point precision flag for af::array More...
Protected Attributes inherited from NeuralEngine::MachineLearning::GPNode< Scalar >
std::vector< std::shared_ptr< GPNode< Scalar > > > | mChild
GPNode< Scalar > * | mParent

## Detailed Description

template<typename Scalar>
class NeuralEngine::MachineLearning::GPModels::AEP::SGPLVM< Scalar >

Sparse GPLVM via Approximated Expectation Propagation (AEP).

Instead of taking one Gaussian portion out to form the cavity, we take out a fraction defined by the parameter \(\alpha\), which can also be seen as a ratio parameter between VFE and PowerEp with FITC approximation.

GPLVM are the nonlinear dual version of probabilistic PCA, where a low dimensional latent variable \(\mathbf{X}=[\mathbf{x}_1,...,\mathbf{x}_N]^T$\f is mapped onto a high dimensional data variable \)\mathbf{Y}=[\mathbf{y}_1,...,\mathbf{y}_N]^T$\f via prior mapping function \(f(\mathbf{x})$\f. The difference to normal GPs is the uncertainty of \)\mathbf{X}$\f, which will be initialized via PCA and optimized during learning. To avoid memory issues for larger data sets, the algorithm takes use of sparse approximation techniques.

Sparse approximations are used for larger data sets to reduce memory size and computational complexity. This is done by introducing a subset of inducing points or pseudo inputs to approximate the full set. The inversion of the kernel matrix depends only on those points and reduces the computational complexity from \(O(N^3)$\) to $$O(k^2N)$\f, where \(k\) is the number of inducing points and \(N\) the length of the data set.

References:

- Bui, T. D. (2018). Efficient Deterministic Approximate Bayesian Inference for Gaussian Process models (Doctoral thesis). https://doi.org/10.17863/CAM.20913

, 24.11.2019.

Definition at line 66 of file FgAEPSparseGPLVM.h.

## Constructor & Destructor Documentation

## ◆ SGPLVM() [1/2]

template<typename Scalar >

NeuralEngine::MachineLearning::GPModels::AEP::SGPLVM< Scalar >::SGPLVM | ( | const af::array & | Y,
|  | int | latentDimension,
|  | int | numInducing = `200`,
|  | Scalar | alpha = `1.0`,
|  | Scalar | priorMean = `0.0`,
|  | Scalar | priorVariance = `1.0`,
|  | PropagationMode | probMode = `PropagationMode::MomentMatching`,
|  | LogLikType | lType = `LogLikType::Gaussian`,
|  | XInit | emethod = `XInit::pca`
| ) |  |

Constructor.

, 21.03.2018.

Parameters

  Y | The training data.
  latentDimension | The latent dimension.
  priorMean | (Optional) The prior mean.
  priorVariance | (Optional) The prior variance.
  numInducing | (Optional) Number of inducing points.
  alpha | (Optional) The fraction parameter.
  lType | (Optional) The likelihood or objective type.
  emethod | (Optional) The initialisation method of the latent variable.

## ◆ SGPLVM() [2/2]

template<typename Scalar >

NeuralEngine::MachineLearning::GPModels::AEP::SGPLVM< Scalar >::SGPLVM | ( |  | ) |

Default constructor.

Hmetal T, 21/03/2021.

## Member Function Documentation

## ◆ Function()

template<typename Scalar >

virtual Scalar NeuralEngine::MachineLearning::GPModels::AEP::SGPLVM< Scalar >::Function | ( | const af::array & | x,
|  | af::array & | outGradient
| ) |  |
| override virtual

Cost function the given parameter inputs.

Hmetal T, 29.11.2017.

Parameters

  x | The parameters to be optimized.
  outGradient | [in,out] The out gradient.

Returns
  A Scalar.

Reimplemented from NeuralEngine::MachineLearning::IModel< Scalar >.

## ◆ CavityLatents()

template<typename Scalar >

void NeuralEngine::MachineLearning::GPModels::AEP::SGPLVM< Scalar >::CavityLatents | ( | af::array & | mx,
|  | af::array & | vx
| ) |  |
| protected

Computes the cavity distribution of the latent points.

Hmetal T, 09/12/2019.

Parameters

  mx | [in,out] The mx.
  vx | [in,out] The vx.

## ◆ CavityGradientLatents()

template<typename Scalar >

af::array NeuralEngine::MachineLearning::GPModels::AEP::SGPLVM< Scalar >::CavityGradientLatents | ( | const af::array & | dmx,
|  | const af::array & | dvx,
|  | const af::array & | m,
|  | const af::array & | v
| ) |  |
| protected

Gradient of the cavity distribution w.r.t. latent points.

HmetalT, 09/12/2019.

Parameters

  dmx | The dmx.
  dvx | The dvx.
  m | An af::array to process.
  v | An af::array to process.

Returns
  An af::array.

## ◆ ComputePhiLatents()

template<typename Scalar >

Scalar NeuralEngine::MachineLearning::GPModels::AEP::SGPLVM< Scalar >::ComputePhiLatents | ( | const af::array & | mx,
|  | const af::array & | vx,
|  | af::array * | dmx = `nullptr`,
|  | af::array * | dvx = `nullptr`
| ) |  |
| protected

Calculates the likelihood contribution of the latent points and its gradients.

Hmetal T, 09/12/2019.

Parameters

  mx | The mean.
  vx | The variance.
  dmx | [in,out] The gradient of the mean.
  dvx | [in,out] The gradient of the variance.

Returns
  The calculated phi contribution.

## ◆ serialize()

template<typename Scalar >
template<class Archive >

void NeuralEngine::MachineLearning::GPModels::AEP::SGPLVM< Scalar >::serialize | ( | Archive & | ar,
|  | unsigned int | version
| ) |  |
| inline private

Definition at line 153 of file FgAEPSparseGPLVM.h.

## Friends And Related Function Documentation

## ◆ boost::serialization::access

template<typename Scalar >

friend class boost::serialization::access
| friend

Definition at line 150 of file FgAEPSparseGPLVM.h.

## Member Data Documentation

## ◆ dAlpha

template<typename Scalar >

Scalar NeuralEngine::MachineLearning::GPModels::AEP::SGPLVM< Scalar >::dAlpha
| private

fraction parameter

Definition at line 147 of file FgAEPSparseGPLVM.h.

## ◆ pMode

template<typename Scalar >

PropagationMode NeuralEngine::MachineLearning::GPModels::AEP::SGPLVM< Scalar >::pMode
| private

Definition at line 148 of file FgAEPSparseGPLVM.h.

The documentation for this class was generated from the following file:

- FgAEPSparseGPLVM.h