NeuralEngine A Game Engine with embeded Machine Learning algorithms based on Gaussian Processes. |

Public Member Functions | Protected Member Functions | Private Member Functions | Private Attributes | Friends | List of all members
NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar > Class Template Reference

Sparse GPSSM via Approximated Expectation Propagation (AEP). More...

`#include <FgAEPSparseGPSSM.h>`

Inheritance diagram for NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >:
[legend]
Collaboration diagram for NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >:
[legend]

## Public Member Functions
| SGPSSM (const af::array &Y, int latentDimension, int numInducing=200, Scalar alpha=1.0, Scalar priorMean=0.0, Scalar priorVariance=1.0, af::array xControl=af::array(), PropagationMode probMode=PropagationMode::MonteCarlo, LogLikType lType=LogLikType::Gaussian, bool GPemission=false, bool controlToEmiss=false, XInit emethod=XInit::pca)
| Constructor. More...
virtual Scalar | Function (const af::array &x, af::array &outGradient) override
| Cost function the given parameter inputs. More...
Public Member Functions inherited from NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >
| SparseGPSSMBaseModel (const af::array &Y, int latentDimension, int numInducing=200, Scalar priorMean=0.0, Scalar priorVariance=1.0, af::array &xControl=af::array(), PropagationMode probMode=PropagationMode::MomentMatching, LogLikType lType=LogLikType::Gaussian, bool GPemission=false, bool controlToEmiss=true, XInit emethod=XInit::pca)
| Constructor. More...
virtual | ~SparseGPSSMBaseModel ()
| Destructor. More...
virtual bool | Init () override
| Initializes the model. More...
virtual void | PredictF (const af::array &testInputs, af::array &mf, af::array &vf) override
| Predict noise free functions values \(\mathbf{F}_*\). More...
virtual void | PredictY (const af::array &testInputs, af::array &my, af::array &vy) override
| Prediction of test outputs \(\mathbf{Y}_*\). More...
virtual void | PredictForward (int numTimeSamples, af::array &my, af::array &vy, af::array *mx=nullptr, af::array *vx=nullptr)
| Noise-free Forward prediction. More...
virtual void | PosteriorData (af::array &my, af::array &vy)
| Get posterior distribution of latent variables /f$\mathbf{X}/f$. More...
virtual int | GetNumParameters () override
| Gets number of parameters. More...
virtual void | SetParameters (const af::array &param) override
| Sets the parameters for each optimization iteration. More...
virtual af::array | GetParameters () override
| Gets the parameters for each optimization iteration. More...
virtual void | UpdateParameters () override
| Updates the parameters. More...
virtual void | FixKernelParameters (bool isfixed)
| Sets fixation for hyperparameters. More...
virtual void | FixInducing (bool isfixed)
| Set fixation for inducing inputs. More...
Public Member Functions inherited from NeuralEngine::MachineLearning::GPModels::GPSSBaseModel< Scalar >
| GPSSBaseModel (const af::array &Y, int latentDimension, Scalar priorMean=0.0, Scalar priorVariance=1.0, af::array &xControl=af::array(), PropagationMode probMode=PropagationMode::MomentMatching, LogLikType lType=LogLikType::Gaussian, bool GPemission=true, bool controlToEmiss=true, XInit emethod=XInit::pca)
| Constructor. More...
| GPSSBaseModel ()
| Default constructor. More...
virtual | ~GPSSBaseModel ()
| Destructor. More...
virtual void | Optimise (OptimizerType method=L_BFGS, Scalar tol=0.0, bool reinit_hypers=true, int maxiter=1000, int mb_size=0, LineSearchType lsType=MoreThuente, bool disp=true, int *cycle=nullptr) override
| Optimizes the model parameters for best fit. More...
virtual bool | Init (af::array &mx)
| Initializes the model. More...
virtual void | PosteriorLatents (af::array &mx, af::array &vx)
| Get posterior distribution of latent variables /f$\mathbf{X}/f$. More...
virtual int | GetNumParameters () override
| Gets number of parameters. More...
virtual void | SetParameters (const af::array &param) override
| Sets the parameters for each optimization iteration. More...
virtual af::array | GetParameters () override
| Gets the parameters for each optimization iteration. More...
void | UpdateDynamicIndexes ()
| Updates the dynamic indexes. More...
void | GetLatents (af::array &mx, af::array &vx)
virtual void | AddWindowData (af::array data)
Public Member Functions inherited from NeuralEngine::MachineLearning::GPModels::GPBaseModel< Scalar >
| GPBaseModel (const af::array &Y, LogLikType lType=LogLikType::Gaussian, ModelType mtype=ModelType::GPR)
| Constructor. More...
| GPBaseModel ()
| Default Constructor. More...
virtual | ~GPBaseModel ()
| Destructor. More...
virtual void | Optimise (OptimizerType method=L_BFGS, Scalar tol=0.0, bool reinit_hypers=true, int maxiter=1000, int mb_size=0, LineSearchType lsType=MoreThuente, bool disp=true, int *cycle=nullptr)
| Optimizes the model parameters for best fit. More...
virtual bool | Init ()
| Initializes the model. More...
virtual void | PredictF (const af::array &testInputs, af::array &mf, af::array &vf)
| Predict noise free functions values \(\mathbf{F}_*\). More...
virtual void | PredictY (const af::array &testInputs, af::array &my, af::array &vy)
| Prediction of test outputs \(\mathbf{Y}_*\). More...
virtual void | SampleY (const af::array inputs, int numSamples, af::array &outFunctions)
| Generate function samples from posterior. More...
virtual void | AddData (const af::array Ytrain)
| Adds training data to the model. More...
af::array | GetTrainingData ()
| Gets the training data set Y. More...
void | SetTrainingData (af::array &data)
| Sets training data Y. More...
virtual int | GetNumParameters ()
| Gets number of parameters. More...
virtual void | SetParameters (const af::array &param)
| Sets the parameters for each optimization iteration. More...
virtual af::array | GetParameters ()
| Gets the parameters for each optimization iteration. More...
virtual void | UpdateParameters ()
| Updates the parameters. More...
virtual void | FixLikelihoodParameters (bool isfixed)
| Sets the likelihood parameters to be fixed or not for optimization. More...
void | SetSegments (af::array segments)
| Sets fixation for hyperparameters. More...
af::array | GetSegments ()
| Gets the start index array for the sequences. More...
Public Member Functions inherited from NeuralEngine::MachineLearning::IModel< Scalar >
virtual Scalar | Function (const af::array &x, af::array &outGradient)
| Cost function the given x inputs. More...
virtual int | GetNumParameters ()=0
| Gets number of parameters to be optimized. More...
virtual void | SetParameters (const af::array &param)=0
| Sets the parameters for each optimization iteration. More...
virtual af::array | GetParameters ()=0
| Gets the parameters for each optimization iteration. More...
virtual void | UpdateParameters ()=0
| Updates the parameters. More...
int | GetDataLenght ()
| Gets data lenght. More...
int | GetDataDimensionality ()
| Gets data dimensionality. More...
ModelType | GetModelType ()
| Gets model type. More...
virtual void | SetBatchSize (int size)
| Sets batch size. More...
int | GetBatchSize ()
| Gets batch size. More...
void | SetIndexes (af::array &indexes)
| Sets the batch indexes. More...
Public Member Functions inherited from NeuralEngine::MachineLearning::GPNode< Scalar >
| GPNode ()
| Default constructor. More...
virtual | ~GPNode ()
| Destructor. More...
int | GetNumChildren () const
| Gets the number of children of this item. More...
int | AttachChild (std::shared_ptr< GPNode< Scalar > > const &child)
| Attaches a child. More...
int | DetachChild (std::shared_ptr< GPNode< Scalar > > const &child)
| Detaches a child. More...
std::shared_ptr< GPNode< Scalar > > | DetachChildAt (int i)
| Detaches a child at index. More...
void | DetachAllChildren ()
| Detach all children from this node. More...
std::shared_ptr< GPNode< Scalar > > | SetChild (int i, std::shared_ptr< GPNode< Scalar > > const &child)
| Sets a child. More...
std::shared_ptr< GPNode< Scalar > > | GetChild (int i)
| Gets a child at index. More...
GPNode< Scalar > * | GetParent ()
| Access to the parent object, which is null for the root of the hierarchy. More...
void | SetParent (GPNode< Scalar > *parent)
| Access to the parent object. Node calls this during attach/detach of children. More...

## Protected Member Functions
virtual void | CavityLatents (af::array &mcav, af::array &vcav, af::array &cav1, af::array &cav2)
| Computes the cavity distribution. More...
Scalar | ComputeTiltedTransition (const af::array &mprob, const af::array &vprob, const af::array &mcav_t1, const af::array &vcav_t1, Scalar scaleLogZDyn, af::array &dlogZ_dmProb, af::array &dlogZ_dvProb, af::array &dlogZ_dmt, af::array &dlogZ_dvt, Scalar &dlogZ_sn)
| Calculates the tilted transition. More...
virtual af::array | PosteriorGradientLatents ()
| Posterior gradient w.r.t \(\mathbf{X}\). More...
virtual af::array | CavityGradientLatents (const af::array &cav1, const af::array &cav2)
| Cavity gradient w.r.t \(\mathbf{X}\). More...
virtual af::array | LogZGradientLatents (const af::array &cav1, const af::array &cav2, const af::array &dmcav_up, const af::array &dvcav_up, const af::array &dmcav_prev, const af::array &dvcav_prev, const af::array &dmcav_next, const af::array &dvcav_next)
| LogZ gradient w.r.t \(\mathbf{X}\). More...
virtual Scalar | ComputePhiPriorLatents ()
| Calculates the phi prior. More...
virtual Scalar | ComputePhiCavityLatents ()
| Calculates the phi cavity. More...
virtual Scalar | ComputePhiPosteriorLatents ()
| Calculates the phi posterior. More...
Protected Member Functions inherited from NeuralEngine::MachineLearning::GPModels::GPSSBaseModel< Scalar >
virtual af::array | PosteriorGradientLatents (const af::array &dmx, const af::array &dvx)
| Posterior gradient of latent inputs /f$\mathbf{X}/f$. More...
virtual void | UpdateParametersInternal ()
| Updates the parameters. More...
Protected Member Functions inherited from NeuralEngine::MachineLearning::IModel< Scalar >
| IModel (int numData, int numDimension, ModelType type)
| Constructor. More...

## Private Member Functions
template<class Archive >
void | serialize (Archive &ar, unsigned int version)

## Private Attributes
Scalar | dAlpha
| fraction parameter More...

## Friends
class | boost::serialization::access

## Additional Inherited Members
Protected Attributes inherited from NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >
int | ik
| number of inducing inputs More...
SparseGPBaseLayer< Scalar > * | dynLayer
| sparse Gaussian Process dynamic layer More...
SparseGPBaseLayer< Scalar > * | gpEmissLayer
| sparse Gaussian Process emission layer More...
GaussEmission< Scalar > * | gaussEmissLayer
| Gaussian distibution emission layer. More...
Protected Attributes inherited from NeuralEngine::MachineLearning::GPModels::GPSSBaseModel< Scalar >
int | iq
| latent dimension More...
int | iDControlEmiss
int | iDControlDyn
Scalar | dSn
bool | bGPemission
bool | bControlToEmiss
Scalar | dPriorMean
| prior mean More...
Scalar | dPriorVariance
| prior variance More...
Scalar | dPriorX1
| prior /f$x_1/f$ More...
Scalar | dPriorX2
| prior /f$x_2/f$ More...
af::array | afFactorX1
| natural parameter factor 1 for latent variable More...
af::array | afFactorX2
| natural parameter factor 2 for latent variable More...
af::array | afPosteriorX1
| posterior natural parameter 1 for latent variable More...
af::array | afPosteriorX2
| posterior natural parameter 2 for latent variable More...
af::array | afXControl
af::array | afDynIndexes
af::array | afPriorX1
| prior /f$x_1/f$ More...
af::array | afPriorX2
| prior /f$x_2/f$ More...
af::array | afPriorX1Cav
| prior /f$x_1/f$ More...
af::array | afPriorX2Cav
| prior /f$x_2/f$ More...
af::array | afPriorMean
| prior mean for hierarchy mode More...
af::array | afPriorVariance
| prior variance for hierarchy mode More...
af::array | afPriorMeanCav
| prior mean for hierarchy mode More...
af::array | afPriorVarianceCav
| prior variance for hierarchy mode More...
af::array | afGradMean
| prior mean gradient for hiersrchy mode More...
af::array | afGradVariance
| prior variance gradient for hiersrchy mode More...
af::array | afGradMeanCav
| prior mean gradient for hiersrchy mode More...
af::array | afGradVarianceCav
| prior variance gradient for hiersrchy mode More...
af::array | afLatentGradientX
| top down gradient More...
XInit | eEmMethod
PropagationMode | pMode
Protected Attributes inherited from NeuralEngine::MachineLearning::GPModels::GPBaseModel< Scalar >
bool | bInit
| check if model is initialized More...
af::array | afY
| training dataset, mean substracted More...
af::array | afBias
| the bias More...
af::array | afSegments
| Index of starting positions for all trials. More...
LikelihoodBaseLayer< Scalar > * | likLayer
| liklihood layer More...
Protected Attributes inherited from NeuralEngine::MachineLearning::IModel< Scalar >
ModelType | mType
int | iN
| dataset length More...
int | iD
| dataset dimension More...
int | iBatchSize
| size of the batch More...
af::array | afIndexes
| indexes of /f$\mathbf{X}/f$ for batch learning More...
af::dtype | m_dType
| floating point precision flag for af::array More...
Protected Attributes inherited from NeuralEngine::MachineLearning::GPNode< Scalar >
std::vector< std::shared_ptr< GPNode< Scalar > > > | mChild
GPNode< Scalar > * | mParent

## Detailed Description

template<typename Scalar>
class NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >

Sparse GPSSM via Approximated Expectation Propagation (AEP).

Instead of taking one Gaussian portion out to form the cavity, we take out a fraction defined by the parameter \(\alpha\), which can also be seen as a ratio parameter between VFE and PowerEp with FITC approximation.

The lack of an additional approximation for the latent function \(f\) in this EP scheme results in a prohibitive computational complexity of \(O(T^3)\). The approach proposed in this class, in contrast, employs Power EP to provide approximate Bayesian estimates for both \(f\) and \(\mathbf{x}\) simultaneously in a computationally and analytically tractable manner. Importantly, Power EP offers a flexible approximate inference framework which has EP and structured variational inference (VI) as special cases.

References:

- Bui, T. D. (2018). Efficient Deterministic Approximate Bayesian Inference for Gaussian Process models (Doctoral thesis). https://doi.org/10.17863/CAM.20913

, 24.11.2019.

Definition at line 56 of file FgAEPSparseGPSSM.h.

## Constructor & Destructor Documentation

## ◆ SGPSSM()

template<typename Scalar >

NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >::SGPSSM | ( | const af::array & | Y,
|  | int | latentDimension,
|  | int | numInducing = `200`,
|  | Scalar | alpha = `1.0`,
|  | Scalar | priorMean = `0.0`,
|  | Scalar | priorVariance = `1.0`,
|  | af::array | xControl = `af::array()`,
|  | PropagationMode | probMode = `PropagationMode::MonteCarlo`,
|  | LogLikType | lType = `LogLikType::Gaussian`,
|  | bool | GPemission = `false`,
|  | bool | controlToEmiss = `false`,
|  | XInit | emethod = `XInit::pca`
| ) |  |

Constructor.

Hmetal T, 04/05/2020.

Parameters

  Y | Observation data.
  latentDimension | The latent dimension.
  numInducing | (Optional) Number of inducings inputs.
  priorMean | (Optional) The prior mean.
  priorVariance | (Optional) The prior variance.
  xControl | [in,out] (Optional) The control.
  lType | (Optional) The likelihood type.
  GPemission | (Optional) True to non-linear emission function.
  controlToEmiss | (Optional) True to control to emiss.
  emethod | (Optional) The embed method.

## Member Function Documentation

## ◆ Function()

template<typename Scalar >

virtual Scalar NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >::Function | ( | const af::array & | x,
|  | af::array & | outGradient
| ) |  |
| override virtual

Cost function the given parameter inputs.

Hmetal T, 29.11.2017.

Parameters

  x | The parameters to be optimized.
  outGradient | [in,out] The out gradient.

Returns
  A Scalar.

Reimplemented from NeuralEngine::MachineLearning::IModel< Scalar >.

## ◆ CavityLatents()

template<typename Scalar >

virtual void NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >::CavityLatents | ( | af::array & | mcav,
|  | af::array & | vcav,
|  | af::array & | cav1,
|  | af::array & | cav2
| ) |  |
| protected virtual

Computes the cavity distribution.

Hmetal T, 11/05/2020.

Parameters

  mcav | [in,out] The mcav.
  vcav | [in,out] The vcav.
  cav1 | [in,out] The first cav.
  cav2 | [in,out] The second cav.

## ◆ ComputeTiltedTransition()

template<typename Scalar >

Scalar NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >::ComputeTiltedTransition | ( | const af::array & | mprob,
|  | const af::array & | vprob,
|  | const af::array & | mcav_t1,
|  | const af::array & | vcav_t1,
|  | Scalar | scaleLogZDyn,
|  | af::array & | dlogZ_dmProb,
|  | af::array & | dlogZ_dvProb,
|  | af::array & | dlogZ_dmt,
|  | af::array & | dlogZ_dvt,
|  | Scalar & | dlogZ_sn
| ) |  |
| protected

Calculates the tilted transition.

Hmetal T, 11/05/2020.

Parameters

  mprob | The mprob.
  vprob | The vprob.
  mcav_t1 | The first mcav t.
  vcav_t1 | The first vcav t.
  scaleLogZDyn | The scale log z coordinate dynamic.
  dmProb | [in,out] The dm prob.
  dvProb | [in,out] The dv prob.
  dmt | [in,out] The dmt.
  dvt | [in,out] The dvt.

Returns
  The calculated tilted transition.

## ◆ PosteriorGradientLatents()

template<typename Scalar >

virtual af::array NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >::PosteriorGradientLatents | ( |  | ) |
| protected virtual

Posterior gradient w.r.t \(\mathbf{X}\).

Hmetal T, 11/05/2020.

Returns
  An af::array.

## ◆ CavityGradientLatents()

template<typename Scalar >

virtual af::array NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >::CavityGradientLatents | ( | const af::array & | cav1,
|  | const af::array & | cav2
| ) |  |
| protected virtual

Cavity gradient w.r.t \(\mathbf{X}\).

Hmetal T, 11/05/2020.

Parameters

  cav1 | The first cav.
  cav2 | The second cav.

Returns
  An af::array.

## ◆ LogZGradientLatents()

template<typename Scalar >

virtual af::array NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >::LogZGradientLatents | ( | const af::array & | cav1,
|  | const af::array & | cav2,
|  | const af::array & | dmcav_up,
|  | const af::array & | dvcav_up,
|  | const af::array & | dmcav_prev,
|  | const af::array & | dvcav_prev,
|  | const af::array & | dmcav_next,
|  | const af::array & | dvcav_next
| ) |  |
| protected virtual

LogZ gradient w.r.t \(\mathbf{X}\).

Hmetal T, 11/05/2020.

Parameters

  cav1 | The first cav.
  cav2 | The second cav.
  dmcav_up | The dmcav up.
  dvcav_up | The dvcav up.
  dmcav_prev | The dmcav previous.
  dvcav_prev | The dvcav previous.
  dmcav_next | The dmcav next.
  dvcav_next | The dvcav next.

Returns
  An af::array.

## ◆ ComputePhiPriorLatents()

template<typename Scalar >

virtual Scalar NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >::ComputePhiPriorLatents | ( |  | ) |
| protected virtual

Calculates the phi prior.

Hmetal T, 11/05/2020.

Returns
  The calculated phi prior latents.

## ◆ ComputePhiCavityLatents()

template<typename Scalar >

virtual Scalar NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >::ComputePhiCavityLatents | ( |  | ) |
| protected virtual

Calculates the phi cavity.

Hmetal T, 11/05/2020.

Returns
  The calculated phi cavity latents.

## ◆ ComputePhiPosteriorLatents()

template<typename Scalar >

virtual Scalar NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >::ComputePhiPosteriorLatents | ( |  | ) |
| protected virtual

Calculates the phi posterior.

Hmetal T, 11/05/2020.

Returns
  The calculated phi posterior latents.

## ◆ serialize()

template<typename Scalar >
template<class Archive >

void NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >::serialize | ( | Archive & | ar,
|  | unsigned int | version
| ) |  |
| inline private

Definition at line 203 of file FgAEPSparseGPSSM.h.

## Friends And Related Function Documentation

## ◆ boost::serialization::access

template<typename Scalar >

friend class boost::serialization::access
| friend

Definition at line 200 of file FgAEPSparseGPSSM.h.

## Member Data Documentation

## ◆ dAlpha

template<typename Scalar >

Scalar NeuralEngine::MachineLearning::GPModels::AEP::SGPSSM< Scalar >::dAlpha
| private

fraction parameter

Definition at line 198 of file FgAEPSparseGPSSM.h.

The documentation for this class was generated from the following file:

- FgAEPSparseGPSSM.h