NeuralEngine A Game Engine with embeded Machine Learning algorithms based on Gaussian Processes. |

Public Member Functions | Protected Attributes | Private Member Functions | Friends | List of all members
NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar > Class Template Reference

Base class for all sparse GPSSM models. More...

`#include <FgSparseGPSSMBaseModel.h>`

Inheritance diagram for NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >:
[legend]
Collaboration diagram for NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >:
[legend]

## Public Member Functions
| SparseGPSSMBaseModel (const af::array &Y, int latentDimension, int numInducing=200, Scalar priorMean=0.0, Scalar priorVariance=1.0, af::array &xControl=af::array(), PropagationMode probMode=PropagationMode::MomentMatching, LogLikType lType=LogLikType::Gaussian, bool GPemission=false, bool controlToEmiss=true, XInit emethod=XInit::pca)
| Constructor. More...
virtual | ~SparseGPSSMBaseModel ()
| Destructor. More...
virtual bool | Init () override
| Initializes the model. More...
virtual void | PredictF (const af::array &testInputs, af::array &mf, af::array &vf) override
| Predict noise free functions values \(\mathbf{F}_*\). More...
virtual void | PredictY (const af::array &testInputs, af::array &my, af::array &vy) override
| Prediction of test outputs \(\mathbf{Y}_*\). More...
virtual void | PredictForward (int numTimeSamples, af::array &my, af::array &vy, af::array *mx=nullptr, af::array *vx=nullptr)
| Noise-free Forward prediction. More...
virtual void | PosteriorData (af::array &my, af::array &vy)
| Get posterior distribution of latent variables /f$\mathbf{X}/f$. More...
virtual int | GetNumParameters () override
| Gets number of parameters. More...
virtual void | SetParameters (const af::array &param) override
| Sets the parameters for each optimization iteration. More...
virtual af::array | GetParameters () override
| Gets the parameters for each optimization iteration. More...
virtual void | UpdateParameters () override
| Updates the parameters. More...
virtual void | FixKernelParameters (bool isfixed)
| Sets fixation for hyperparameters. More...
virtual void | FixInducing (bool isfixed)
| Set fixation for inducing inputs. More...
Public Member Functions inherited from NeuralEngine::MachineLearning::GPModels::GPSSBaseModel< Scalar >
| GPSSBaseModel (const af::array &Y, int latentDimension, Scalar priorMean=0.0, Scalar priorVariance=1.0, af::array &xControl=af::array(), PropagationMode probMode=PropagationMode::MomentMatching, LogLikType lType=LogLikType::Gaussian, bool GPemission=true, bool controlToEmiss=true, XInit emethod=XInit::pca)
| Constructor. More...
| GPSSBaseModel ()
| Default constructor. More...
virtual | ~GPSSBaseModel ()
| Destructor. More...
virtual void | Optimise (OptimizerType method=L_BFGS, Scalar tol=0.0, bool reinit_hypers=true, int maxiter=1000, int mb_size=0, LineSearchType lsType=MoreThuente, bool disp=true, int *cycle=nullptr) override
| Optimizes the model parameters for best fit. More...
virtual bool | Init (af::array &mx)
| Initializes the model. More...
virtual void | PosteriorLatents (af::array &mx, af::array &vx)
| Get posterior distribution of latent variables /f$\mathbf{X}/f$. More...
virtual int | GetNumParameters () override
| Gets number of parameters. More...
virtual void | SetParameters (const af::array &param) override
| Sets the parameters for each optimization iteration. More...
virtual af::array | GetParameters () override
| Gets the parameters for each optimization iteration. More...
void | UpdateDynamicIndexes ()
| Updates the dynamic indexes. More...
void | GetLatents (af::array &mx, af::array &vx)
virtual void | AddWindowData (af::array data)
Public Member Functions inherited from NeuralEngine::MachineLearning::GPModels::GPBaseModel< Scalar >
| GPBaseModel (const af::array &Y, LogLikType lType=LogLikType::Gaussian, ModelType mtype=ModelType::GPR)
| Constructor. More...
| GPBaseModel ()
| Default Constructor. More...
virtual | ~GPBaseModel ()
| Destructor. More...
virtual void | Optimise (OptimizerType method=L_BFGS, Scalar tol=0.0, bool reinit_hypers=true, int maxiter=1000, int mb_size=0, LineSearchType lsType=MoreThuente, bool disp=true, int *cycle=nullptr)
| Optimizes the model parameters for best fit. More...
virtual bool | Init ()
| Initializes the model. More...
virtual void | PredictF (const af::array &testInputs, af::array &mf, af::array &vf)
| Predict noise free functions values \(\mathbf{F}_*\). More...
virtual void | PredictY (const af::array &testInputs, af::array &my, af::array &vy)
| Prediction of test outputs \(\mathbf{Y}_*\). More...
virtual void | SampleY (const af::array inputs, int numSamples, af::array &outFunctions)
| Generate function samples from posterior. More...
virtual void | AddData (const af::array Ytrain)
| Adds training data to the model. More...
af::array | GetTrainingData ()
| Gets the training data set Y. More...
void | SetTrainingData (af::array &data)
| Sets training data Y. More...
virtual int | GetNumParameters ()
| Gets number of parameters. More...
virtual void | SetParameters (const af::array &param)
| Sets the parameters for each optimization iteration. More...
virtual af::array | GetParameters ()
| Gets the parameters for each optimization iteration. More...
virtual void | UpdateParameters ()
| Updates the parameters. More...
virtual void | FixLikelihoodParameters (bool isfixed)
| Sets the likelihood parameters to be fixed or not for optimization. More...
void | SetSegments (af::array segments)
| Sets fixation for hyperparameters. More...
af::array | GetSegments ()
| Gets the start index array for the sequences. More...
Public Member Functions inherited from NeuralEngine::MachineLearning::IModel< Scalar >
virtual Scalar | Function (const af::array &x, af::array &outGradient)
| Cost function the given x inputs. More...
virtual int | GetNumParameters ()=0
| Gets number of parameters to be optimized. More...
virtual void | SetParameters (const af::array &param)=0
| Sets the parameters for each optimization iteration. More...
virtual af::array | GetParameters ()=0
| Gets the parameters for each optimization iteration. More...
virtual void | UpdateParameters ()=0
| Updates the parameters. More...
int | GetDataLenght ()
| Gets data lenght. More...
int | GetDataDimensionality ()
| Gets data dimensionality. More...
ModelType | GetModelType ()
| Gets model type. More...
virtual void | SetBatchSize (int size)
| Sets batch size. More...
int | GetBatchSize ()
| Gets batch size. More...
void | SetIndexes (af::array &indexes)
| Sets the batch indexes. More...
Public Member Functions inherited from NeuralEngine::MachineLearning::GPNode< Scalar >
| GPNode ()
| Default constructor. More...
virtual | ~GPNode ()
| Destructor. More...
int | GetNumChildren () const
| Gets the number of children of this item. More...
int | AttachChild (std::shared_ptr< GPNode< Scalar > > const &child)
| Attaches a child. More...
int | DetachChild (std::shared_ptr< GPNode< Scalar > > const &child)
| Detaches a child. More...
std::shared_ptr< GPNode< Scalar > > | DetachChildAt (int i)
| Detaches a child at index. More...
void | DetachAllChildren ()
| Detach all children from this node. More...
std::shared_ptr< GPNode< Scalar > > | SetChild (int i, std::shared_ptr< GPNode< Scalar > > const &child)
| Sets a child. More...
std::shared_ptr< GPNode< Scalar > > | GetChild (int i)
| Gets a child at index. More...
GPNode< Scalar > * | GetParent ()
| Access to the parent object, which is null for the root of the hierarchy. More...
void | SetParent (GPNode< Scalar > *parent)
| Access to the parent object. Node calls this during attach/detach of children. More...

## Protected Attributes
int | ik
| number of inducing inputs More...
SparseGPBaseLayer< Scalar > * | dynLayer
| sparse Gaussian Process dynamic layer More...
SparseGPBaseLayer< Scalar > * | gpEmissLayer
| sparse Gaussian Process emission layer More...
GaussEmission< Scalar > * | gaussEmissLayer
| Gaussian distibution emission layer. More...
Protected Attributes inherited from NeuralEngine::MachineLearning::GPModels::GPSSBaseModel< Scalar >
int | iq
| latent dimension More...
int | iDControlEmiss
int | iDControlDyn
Scalar | dSn
bool | bGPemission
bool | bControlToEmiss
Scalar | dPriorMean
| prior mean More...
Scalar | dPriorVariance
| prior variance More...
Scalar | dPriorX1
| prior /f$x_1/f$ More...
Scalar | dPriorX2
| prior /f$x_2/f$ More...
af::array | afFactorX1
| natural parameter factor 1 for latent variable More...
af::array | afFactorX2
| natural parameter factor 2 for latent variable More...
af::array | afPosteriorX1
| posterior natural parameter 1 for latent variable More...
af::array | afPosteriorX2
| posterior natural parameter 2 for latent variable More...
af::array | afXControl
af::array | afDynIndexes
af::array | afPriorX1
| prior /f$x_1/f$ More...
af::array | afPriorX2
| prior /f$x_2/f$ More...
af::array | afPriorX1Cav
| prior /f$x_1/f$ More...
af::array | afPriorX2Cav
| prior /f$x_2/f$ More...
af::array | afPriorMean
| prior mean for hierarchy mode More...
af::array | afPriorVariance
| prior variance for hierarchy mode More...
af::array | afPriorMeanCav
| prior mean for hierarchy mode More...
af::array | afPriorVarianceCav
| prior variance for hierarchy mode More...
af::array | afGradMean
| prior mean gradient for hiersrchy mode More...
af::array | afGradVariance
| prior variance gradient for hiersrchy mode More...
af::array | afGradMeanCav
| prior mean gradient for hiersrchy mode More...
af::array | afGradVarianceCav
| prior variance gradient for hiersrchy mode More...
af::array | afLatentGradientX
| top down gradient More...
XInit | eEmMethod
PropagationMode | pMode
Protected Attributes inherited from NeuralEngine::MachineLearning::GPModels::GPBaseModel< Scalar >
bool | bInit
| check if model is initialized More...
af::array | afY
| training dataset, mean substracted More...
af::array | afBias
| the bias More...
af::array | afSegments
| Index of starting positions for all trials. More...
LikelihoodBaseLayer< Scalar > * | likLayer
| liklihood layer More...
Protected Attributes inherited from NeuralEngine::MachineLearning::IModel< Scalar >
ModelType | mType
int | iN
| dataset length More...
int | iD
| dataset dimension More...
int | iBatchSize
| size of the batch More...
af::array | afIndexes
| indexes of /f$\mathbf{X}/f$ for batch learning More...
af::dtype | m_dType
| floating point precision flag for af::array More...
Protected Attributes inherited from NeuralEngine::MachineLearning::GPNode< Scalar >
std::vector< std::shared_ptr< GPNode< Scalar > > > | mChild
GPNode< Scalar > * | mParent

## Private Member Functions
template<class Archive >
void | serialize (Archive &ar, unsigned int version)

## Friends
class | AEP::SGPLayer< Scalar >
class | PowerEP::SGPLayer< Scalar >
class | GaussEmission< Scalar >
class | boost::serialization::access

## Additional Inherited Members
Protected Member Functions inherited from NeuralEngine::MachineLearning::GPModels::GPSSBaseModel< Scalar >
virtual af::array | PosteriorGradientLatents (const af::array &dmx, const af::array &dvx)
| Posterior gradient of latent inputs /f$\mathbf{X}/f$. More...
virtual void | UpdateParametersInternal ()
| Updates the parameters. More...
Protected Member Functions inherited from NeuralEngine::MachineLearning::IModel< Scalar >
| IModel (int numData, int numDimension, ModelType type)
| Constructor. More...

## Detailed Description

template<typename Scalar>
class NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >

Base class for all sparse GPSSM models.

GP state-space model in which latent variables are continuous and the transition between these variables is non-linear. Due to the non - linearity in the dynamics, inferenceand learning, which involve finding the state transition as well as the latent states themselves, are difficult.The presence of transition and measurement noise, and a typically small number of measurements in practice make the inference task even more challenging. This model subclass can be compactly represented as follows,

\[ p(\mathbf{x}_0) = \mathcal{N}(\mathbf{x}_0; \mu_0, \Sigma_0), \]

\[ p(\mathbf{x}_t | f, \mathbf{x}_{t−1}) = \mathcal{N}(\mathbf{x}_t; f(\mathbf{x}_{t−1}), \mathbf{Q}_x), t = 1 : T, \]

\[ p(\mathbf{y}_t | \mathbf{x}_t) = \mathcal{N}(\mathbf{y}_t; \mathbf{U}\mathbf{x}_t, \mathbf{R}_y), t = 1 : T, \]

where the dynamical noise is assumed Gaussian and the measurement or emission model is assumed to be linear and Gaussian, \(\mathbf{x}\) and \(\mathbf{y}\) are the latent variables and the measurements respectively, \(f\) is a continuous non- linear transition function, and \(T\) is the number of measurement steps.

We assume the non-linear transition function f is a draw from a Gaussian process, or in other words, we place a flexible nonparametric Gaussian process prior over this function, \(p(f) = GP(m(.), k(., .))\). We refer to the resulting model as the Gaussian process state - space model (GPSSM).This model belongs to a plethora of closely related dynamical systems which describe how the state and emission variables evolve using Gaussian processes. The joint probablity of all variablesand observations involved can be written as follows,

\[ p(\mathbf{y}_{1:T}, \mathbf{x}_{0:T}, f | \theta) = p(\mathbf{x}_0)p(f | \theta) \prod_{t=1}^T p(\mathbf{x}_t | f, \mathbf{x}_{t−1}, \theta) p(\mathbf{y}_t | \mathbf{x}_t, \theta), \]

where \(\theta\) includes the kernel hyperparameters, the transition noise, and the emission parameters. These hyperparameters can be found by performing model selection using the log marginal likelihood:

\[L(\theta) = log \int p(\mathbf{y}_{1:T} | \theta) dfd\mathbf{x}_{0:T} p(\mathbf{y}_{1:T}, \mathbf{x}_{0:T}, f | \theta).\]

References:

- Damianou, AC, Titsias, MK, Lawrence, ND (2012). "Variational gaussian process dynamical systems". Advances in Neural Information Processing Systems

, 24.11.2019.

Definition at line 100 of file FgSparseGPSSMBaseModel.h.

## Constructor & Destructor Documentation

## ◆ SparseGPSSMBaseModel()

template<typename Scalar >

NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::SparseGPSSMBaseModel | ( | const af::array & | Y,
|  | int | latentDimension,
|  | int | numInducing = `200`,
|  | Scalar | priorMean = `0.0`,
|  | Scalar | priorVariance = `1.0`,
|  | af::array & | xControl = `af::array()`,
|  | PropagationMode | probMode = `PropagationMode::MomentMatching`,
|  | LogLikType | lType = `LogLikType::Gaussian`,
|  | bool | GPemission = `false`,
|  | bool | controlToEmiss = `true`,
|  | XInit | emethod = `XInit::pca`
| ) |  |

Constructor.

Hmetal T, 04/05/2020.

Parameters

  Y | Observation data.
  latentDimension | The latent dimension.
  numInducing | (Optional) Number of inducings inputs.
  priorMean | (Optional) The prior mean.
  priorVariance | (Optional) The prior variance.
  xControl | [in,out] (Optional) The control.
  lType | (Optional) The likelihood type.
  GPemission | (Optional) True to non-linear emission function.
  controlToEmiss | (Optional) True to control to emiss.
  emethod | (Optional) The embed method.

## ◆ ~SparseGPSSMBaseModel()

template<typename Scalar >

virtual NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::~SparseGPSSMBaseModel | ( |  | ) |
| virtual

Destructor.

Hmetal T, 05/05/2020.

## Member Function Documentation

## ◆ Init()

template<typename Scalar >

virtual bool NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::Init | ( |  | ) |
| override virtual

Initializes the model.

Hmetal T, 29.11.2017.

Returns
  true if it succeeds, false if it fails.

Reimplemented from NeuralEngine::MachineLearning::GPModels::GPBaseModel< Scalar >.

## ◆ PredictF()

template<typename Scalar >

virtual void NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::PredictF | ( | const af::array & | testInputs,
|  | af::array & | mf,
|  | af::array & | vf
| ) |  |
| override virtual

Predict noise free functions values \(\mathbf{F}_*\).

Hmetal T, 05/05/2020.

Parameters

  testInputs | The test inputs.
  mf | [in,out] mean of function values.
  vf | [in,out] The variance of function values.

Reimplemented from NeuralEngine::MachineLearning::GPModels::GPBaseModel< Scalar >.

## ◆ PredictY()

template<typename Scalar >

virtual void NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::PredictY | ( | const af::array & | testInputs,
|  | af::array & | my,
|  | af::array & | vy
| ) |  |
| override virtual

Prediction of test outputs \(\mathbf{Y}_*\).

, 12.06.2018.

Parameters

  my | [in,out] The posterior mean function.
  vy | [in,out] The posterior covariance function.
  testX | [in,out] The test inputs.

Reimplemented from NeuralEngine::MachineLearning::GPModels::GPBaseModel< Scalar >.

## ◆ PredictForward()

template<typename Scalar >

virtual void NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::PredictForward | ( | int | numTimeSamples,
|  | af::array & | my,
|  | af::array & | vy,
|  | af::array * | mx = `nullptr`,
|  | af::array * | vx = `nullptr`
| ) |  |
| virtual

Noise-free Forward prediction.

Hmetal T, 07/05/2020.

Parameters

  numTimeSamples | Number of time samples.
  mf | [in,out] The mean uf \(\matthbf{Y}_*\).
  vf | [in,out] The variance of \(\matthbf{Y}_*\).
  mx | [in,out] (Optional) If non-null, the mx.
  vx | [in,out] (Optional) If non-null, the vx.

## ◆ PosteriorData()

template<typename Scalar >

virtual void NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::PosteriorData | ( | af::array & | my,
|  | af::array & | vy
| ) |  |
| virtual

Get posterior distribution of latent variables /f$\mathbf{X}/f$.

Hmetal T, 09/12/2019.

Parameters

  my | [in,out] The posterior mean of the data.
  vy | [in,out] The posterior variance of the data.

## ◆ GetNumParameters()

template<typename Scalar >

virtual int NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::GetNumParameters | ( |  | ) |
| override virtual

Gets number of parameters.

, 26.06.2018.

Returns
  The number parameters.

Reimplemented from NeuralEngine::MachineLearning::GPModels::GPSSBaseModel< Scalar >.

## ◆ SetParameters()

template<typename Scalar >

virtual void NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::SetParameters | ( | const af::array & | param | ) |
| override virtual

Sets the parameters for each optimization iteration.

, 26.06.2018.

Parameters

  param | The parameter.

Reimplemented from NeuralEngine::MachineLearning::GPModels::GPSSBaseModel< Scalar >.

## ◆ GetParameters()

template<typename Scalar >

virtual af::array NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::GetParameters | ( |  | ) |
| override virtual

Gets the parameters for each optimization iteration.

, 26.06.2018.

Parameters

  param | The parameter.

Reimplemented from NeuralEngine::MachineLearning::GPModels::GPSSBaseModel< Scalar >.

## ◆ UpdateParameters()

template<typename Scalar >

virtual void NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::UpdateParameters | ( |  | ) |
| override virtual

Updates the parameters.

, 26.06.2018.

Reimplemented from NeuralEngine::MachineLearning::GPModels::GPBaseModel< Scalar >.

## ◆ FixKernelParameters()

template<typename Scalar >

virtual void NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::FixKernelParameters | ( | bool | isfixed | ) |
| virtual

Sets fixation for hyperparameters.

Hmetal T, 16/12/2019.

Parameters

  isfixed | True if isfixed.

## ◆ FixInducing()

template<typename Scalar >

virtual void NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::FixInducing | ( | bool | isfixed | ) |
| virtual

Set fixation for inducing inputs.

Hmetal T, 16/12/2019.

Parameters

  isfixed | True if isfixed.

## ◆ serialize()

template<typename Scalar >
template<class Archive >

void NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::serialize | ( | Archive & | ar,
|  | unsigned int | version
| ) |  |
| inline private

Definition at line 256 of file FgSparseGPSSMBaseModel.h.

## Friends And Related Function Documentation

## ◆ AEP::SGPLayer< Scalar >

template<typename Scalar >

friend class AEP::SGPLayer< Scalar >
| friend

Definition at line 245 of file FgSparseGPSSMBaseModel.h.

## ◆ PowerEP::SGPLayer< Scalar >

template<typename Scalar >

friend class PowerEP::SGPLayer< Scalar >
| friend

Definition at line 245 of file FgSparseGPSSMBaseModel.h.

## ◆ GaussEmission< Scalar >

template<typename Scalar >

friend class GaussEmission< Scalar >
| friend

Definition at line 245 of file FgSparseGPSSMBaseModel.h.

## ◆ boost::serialization::access

template<typename Scalar >

friend class boost::serialization::access
| friend

Definition at line 253 of file FgSparseGPSSMBaseModel.h.

## Member Data Documentation

## ◆ ik

template<typename Scalar >

int NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::ik
| protected

number of inducing inputs

Definition at line 242 of file FgSparseGPSSMBaseModel.h.

## ◆ dynLayer

template<typename Scalar >

SparseGPBaseLayer<Scalar>* NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::dynLayer
| protected

sparse Gaussian Process dynamic layer

Definition at line 243 of file FgSparseGPSSMBaseModel.h.

## ◆ gpEmissLayer

template<typename Scalar >

SparseGPBaseLayer<Scalar>* NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::gpEmissLayer
| protected

sparse Gaussian Process emission layer

Definition at line 244 of file FgSparseGPSSMBaseModel.h.

## ◆ gaussEmissLayer

template<typename Scalar >

GaussEmission<Scalar>* NeuralEngine::MachineLearning::GPModels::SparseGPSSMBaseModel< Scalar >::gaussEmissLayer
| protected

Gaussian distibution emission layer.

Definition at line 245 of file FgSparseGPSSMBaseModel.h.

The documentation for this class was generated from the following file:

- FgSparseGPSSMBaseModel.h