Sparse deep GPSSM via Approximated Expectation Propagation (AEP). More...
#include <FgAEPSparseDGPSSM.h>
Public Member Functions | |
SDGPSSM (const af::array &Y, int latentDimension, HiddenLayerDescription description, Scalar alpha=1.0, Scalar priorMean=0.0, Scalar priorVariance=1.0, af::array xControl=af::array(), PropagationMode probMode=PropagationMode::MomentMatching, LogLikType lType=LogLikType::Gaussian, XInit emethod=XInit::pca) | |
Constructor. More... | |
SDGPSSM (const af::array &Y, int latentDimension, std::vector< HiddenLayerDescription > descriptions, Scalar alpha=1.0, Scalar priorMean=0.0, Scalar priorVariance=1.0, af::array xControl=af::array(), PropagationMode probMode=PropagationMode::MomentMatching, LogLikType lType=LogLikType::Gaussian, XInit emethod=XInit::pca) | |
Constructor. More... | |
SDGPSSM () | |
Default Constructor. More... | |
virtual | ~SDGPSSM () |
Destructor. More... | |
virtual Scalar | Function (const af::array &x, af::array &outGradient) override |
Cost function the given parameter inputs. More... | |
![]() | |
SparseDeepGPSSMBaseModel (const af::array &Y, int latentDimension, HiddenLayerDescription description, Scalar priorMean=0.0, Scalar priorVariance=1.0, af::array &xControl=af::array(), PropagationMode probMode=PropagationMode::MomentMatching, LogLikType lType=LogLikType::Gaussian, XInit emethod=XInit::pca) | |
Constructor. More... | |
SparseDeepGPSSMBaseModel (const af::array &Y, int latentDimension, std::vector< HiddenLayerDescription > descriptions, Scalar priorMean=0.0, Scalar priorVariance=1.0, af::array &xControl=af::array(), PropagationMode probMode=PropagationMode::MomentMatching, LogLikType lType=LogLikType::Gaussian, XInit emethod=XInit::pca) | |
Constructor. More... | |
SparseDeepGPSSMBaseModel () | |
Default Constructor. More... | |
virtual | ~SparseDeepGPSSMBaseModel () |
Destructor. More... | |
virtual bool | Init () override |
Initializes the model. More... | |
virtual void | PredictF (const af::array &testInputs, af::array &mf, af::array &vf) override |
Predict noise free functions values \(\mathbf{F}_*\). More... | |
virtual void | PredictForward (int numTimeSamples, af::array &my, af::array &vy, int numSamples=200, af::array *mx=nullptr, af::array *vx=nullptr) |
Noise-free Forward prediction. More... | |
virtual int | GetNumParameters () override |
Generate function samples from posterior. More... | |
virtual int | GetNumGPLayerParameters () |
Gets number gp layer parameters. More... | |
virtual void | SetParameters (const af::array ¶m) override |
Sets the parameters for each optimization iteration. More... | |
virtual af::array | GetParameters () override |
Gets the parameters for each optimization iteration. More... | |
virtual void | UpdateParameters () override |
Updates the parameters. More... | |
virtual int | GetNumLayers () |
Gets number of GP layers. More... | |
virtual std::vector< SparseGPBaseLayer< Scalar > * > | GetGPLayers () |
Gets vector of GP layers. More... | |
virtual void | FixKernelParameters (bool isfixed) |
Sets fixation for hyperparameters. More... | |
![]() | |
GPSSBaseModel (const af::array &Y, int latentDimension, Scalar priorMean=0.0, Scalar priorVariance=1.0, af::array &xControl=af::array(), PropagationMode probMode=PropagationMode::MomentMatching, LogLikType lType=LogLikType::Gaussian, bool GPemission=true, bool controlToEmiss=true, XInit emethod=XInit::pca) | |
Constructor. More... | |
GPSSBaseModel () | |
Default constructor. More... | |
virtual | ~GPSSBaseModel () |
Destructor. More... | |
virtual void | Optimise (OptimizerType method=L_BFGS, Scalar tol=0.0, bool reinit_hypers=true, int maxiter=1000, int mb_size=0, LineSearchType lsType=MoreThuente, bool disp=true, int *cycle=nullptr) override |
Optimizes the model parameters for best fit. More... | |
virtual bool | Init (af::array &mx) |
Initializes the model. More... | |
virtual void | PosteriorLatents (af::array &mx, af::array &vx) |
Get posterior distribution of latent variables /f$\mathbf{X}/f$. More... | |
virtual int | GetNumParameters () override |
Gets number of parameters. More... | |
virtual void | SetParameters (const af::array ¶m) override |
Sets the parameters for each optimization iteration. More... | |
virtual af::array | GetParameters () override |
Gets the parameters for each optimization iteration. More... | |
void | UpdateDynamicIndexes () |
Updates the dynamic indexes. More... | |
void | GetLatents (af::array &mx, af::array &vx) |
virtual void | AddWindowData (af::array data) |
![]() | |
GPBaseModel (const af::array &Y, LogLikType lType=LogLikType::Gaussian, ModelType mtype=ModelType::GPR) | |
Constructor. More... | |
GPBaseModel () | |
Default Constructor. More... | |
virtual | ~GPBaseModel () |
Destructor. More... | |
virtual void | Optimise (OptimizerType method=L_BFGS, Scalar tol=0.0, bool reinit_hypers=true, int maxiter=1000, int mb_size=0, LineSearchType lsType=MoreThuente, bool disp=true, int *cycle=nullptr) |
Optimizes the model parameters for best fit. More... | |
virtual bool | Init () |
Initializes the model. More... | |
virtual void | PredictF (const af::array &testInputs, af::array &mf, af::array &vf) |
Predict noise free functions values \(\mathbf{F}_*\). More... | |
virtual void | PredictY (const af::array &testInputs, af::array &my, af::array &vy) |
Prediction of test outputs \(\mathbf{Y}_*\). More... | |
virtual void | SampleY (const af::array inputs, int numSamples, af::array &outFunctions) |
Generate function samples from posterior. More... | |
virtual void | AddData (const af::array Ytrain) |
Adds training data to the model. More... | |
af::array | GetTrainingData () |
Gets the training data set Y. More... | |
void | SetTrainingData (af::array &data) |
Sets training data Y. More... | |
virtual int | GetNumParameters () |
Gets number of parameters. More... | |
virtual void | SetParameters (const af::array ¶m) |
Sets the parameters for each optimization iteration. More... | |
virtual af::array | GetParameters () |
Gets the parameters for each optimization iteration. More... | |
virtual void | UpdateParameters () |
Updates the parameters. More... | |
virtual void | FixLikelihoodParameters (bool isfixed) |
Sets the likelihood parameters to be fixed or not for optimization. More... | |
void | SetSegments (af::array segments) |
Sets fixation for hyperparameters. More... | |
af::array | GetSegments () |
Gets the start index array for the sequences. More... | |
![]() | |
virtual Scalar | Function (const af::array &x, af::array &outGradient) |
Cost function the given x inputs. More... | |
virtual int | GetNumParameters ()=0 |
Gets number of parameters to be optimized. More... | |
virtual void | SetParameters (const af::array ¶m)=0 |
Sets the parameters for each optimization iteration. More... | |
virtual af::array | GetParameters ()=0 |
Gets the parameters for each optimization iteration. More... | |
virtual void | UpdateParameters ()=0 |
Updates the parameters. More... | |
int | GetDataLenght () |
Gets data lenght. More... | |
int | GetDataDimensionality () |
Gets data dimensionality. More... | |
ModelType | GetModelType () |
Gets model type. More... | |
virtual void | SetBatchSize (int size) |
Sets batch size. More... | |
int | GetBatchSize () |
Gets batch size. More... | |
void | SetIndexes (af::array &indexes) |
Sets the batch indexes. More... | |
![]() | |
GPNode () | |
Default constructor. More... | |
virtual | ~GPNode () |
Destructor. More... | |
int | GetNumChildren () const |
Gets the number of children of this item. More... | |
int | AttachChild (std::shared_ptr< GPNode< Scalar > > const &child) |
Attaches a child. More... | |
int | DetachChild (std::shared_ptr< GPNode< Scalar > > const &child) |
Detaches a child. More... | |
std::shared_ptr< GPNode< Scalar > > | DetachChildAt (int i) |
Detaches a child at index. More... | |
void | DetachAllChildren () |
Detach all children from this node. More... | |
std::shared_ptr< GPNode< Scalar > > | SetChild (int i, std::shared_ptr< GPNode< Scalar > > const &child) |
Sets a child. More... | |
std::shared_ptr< GPNode< Scalar > > | GetChild (int i) |
Gets a child at index. More... | |
GPNode< Scalar > * | GetParent () |
Access to the parent object, which is null for the root of the hierarchy. More... | |
void | SetParent (GPNode< Scalar > *parent) |
Access to the parent object. Node calls this during attach/detach of children. More... | |
Protected Member Functions | |
virtual void | CavityLatents (af::array &mcav, af::array &vcav, af::array &cav1, af::array &cav2) |
Computes the cavity distribution. More... | |
Scalar | ComputeTiltedTransition (const af::array &mprob, const af::array &vprob, const af::array &mcav_t1, const af::array &vcav_t1, Scalar scaleLogZDyn, af::array &dlogZ_dmProb, af::array &dlogZ_dvProb, af::array &dlogZ_dmt, af::array &dlogZ_dvt, Scalar &dlogZ_sn) |
Calculates the tilted transition. More... | |
virtual af::array | PosteriorGradientLatents () |
Posterior gradient w.r.t \(\mathbf{X}\). More... | |
virtual af::array | CavityGradientLatents (const af::array &cav1, const af::array &cav2) |
Cavity gradient w.r.t \(\mathbf{X}\). More... | |
virtual af::array | LogZGradientLatents (const af::array &cav1, const af::array &cav2, const af::array &dmcav_up, const af::array &dvcav_up, const af::array &dmcav_prev, const af::array &dvcav_prev, const af::array &dmcav_next, const af::array &dvcav_next) |
LogZ gradient w.r.t \(\mathbf{X}\). More... | |
virtual Scalar | ComputePhiPriorLatents () |
Calculates the phi prior. More... | |
virtual Scalar | ComputePhiCavityLatents () |
Calculates the phi cavity. More... | |
virtual Scalar | ComputePhiPosteriorLatents () |
Calculates the phi posterior. More... | |
![]() | |
virtual void | PredictForwardMM (int numTimeSamples, af::array &my, af::array &vy, af::array *mx=nullptr, af::array *vx=nullptr) |
virtual void | PredictForwardMC (int numTimeSamples, af::array &my, af::array &vy, int numSamples=200, af::array *mx=nullptr) |
![]() | |
virtual af::array | PosteriorGradientLatents (const af::array &dmx, const af::array &dvx) |
Posterior gradient of latent inputs /f$\mathbf{X}/f$. More... | |
virtual void | UpdateParametersInternal () |
Updates the parameters. More... | |
![]() | |
IModel (int numData, int numDimension, ModelType type) | |
Constructor. More... | |
Private Member Functions | |
template<class Archive > | |
void | serialize (Archive &ar, unsigned int version) |
Private Attributes | |
Scalar | dAlpha |
fraction parameter More... | |
Friends | |
class | boost::serialization::access |
Additional Inherited Members | |
![]() | |
int | iNumLayer |
std::vector< int > | vNumPseudosPerLayer |
std::vector< int > | vSize |
std::vector< HiddenLayerDescription > | vDescription |
SparseGPBaseLayer< Scalar > * | dynLayer |
sparse Gaussian Process dynamic layer More... | |
std::vector< SparseGPBaseLayer< Scalar > * > | gpEmissLayer |
sparse Gaussian Process emission layer More... | |
![]() | |
int | iq |
latent dimension More... | |
int | iDControlEmiss |
int | iDControlDyn |
Scalar | dSn |
bool | bGPemission |
bool | bControlToEmiss |
Scalar | dPriorMean |
prior mean More... | |
Scalar | dPriorVariance |
prior variance More... | |
Scalar | dPriorX1 |
prior /f$x_1/f$ More... | |
Scalar | dPriorX2 |
prior /f$x_2/f$ More... | |
af::array | afFactorX1 |
natural parameter factor 1 for latent variable More... | |
af::array | afFactorX2 |
natural parameter factor 2 for latent variable More... | |
af::array | afPosteriorX1 |
posterior natural parameter 1 for latent variable More... | |
af::array | afPosteriorX2 |
posterior natural parameter 2 for latent variable More... | |
af::array | afXControl |
af::array | afDynIndexes |
af::array | afPriorX1 |
prior /f$x_1/f$ More... | |
af::array | afPriorX2 |
prior /f$x_2/f$ More... | |
af::array | afPriorX1Cav |
prior /f$x_1/f$ More... | |
af::array | afPriorX2Cav |
prior /f$x_2/f$ More... | |
af::array | afPriorMean |
prior mean for hierarchy mode More... | |
af::array | afPriorVariance |
prior variance for hierarchy mode More... | |
af::array | afPriorMeanCav |
prior mean for hierarchy mode More... | |
af::array | afPriorVarianceCav |
prior variance for hierarchy mode More... | |
af::array | afGradMean |
prior mean gradient for hiersrchy mode More... | |
af::array | afGradVariance |
prior variance gradient for hiersrchy mode More... | |
af::array | afGradMeanCav |
prior mean gradient for hiersrchy mode More... | |
af::array | afGradVarianceCav |
prior variance gradient for hiersrchy mode More... | |
af::array | afLatentGradientX |
top down gradient More... | |
XInit | eEmMethod |
PropagationMode | pMode |
![]() | |
bool | bInit |
check if model is initialized More... | |
af::array | afY |
training dataset, mean substracted More... | |
af::array | afBias |
the bias More... | |
af::array | afSegments |
Index of starting positions for all trials. More... | |
LikelihoodBaseLayer< Scalar > * | likLayer |
liklihood layer More... | |
![]() | |
ModelType | mType |
int | iN |
dataset length More... | |
int | iD |
dataset dimension More... | |
int | iBatchSize |
size of the batch More... | |
af::array | afIndexes |
indexes of /f$\mathbf{X}/f$ for batch learning More... | |
af::dtype | m_dType |
floating point precision flag for af::array More... | |
![]() | |
std::vector< std::shared_ptr< GPNode< Scalar > > > | mChild |
GPNode< Scalar > * | mParent |
Sparse deep GPSSM via Approximated Expectation Propagation (AEP).
Instead of taking one Gaussian portion out to form the cavity, we take out a fraction defined by the parameter \(\alpha\), which can also be seen as a ratio parameter between VFE and PowerEp with FITC approximation. This enables deep structures for GPLVM.
GPLVM are the nonlinear dual version of probabilistic PCA, where a low dimensional latent variable \(\mathbf{X}=[\mathbf{x}_1,...,\mathbf{x}_N]^T$\f is mapped onto a
high dimensional data variable \)\mathbf{Y}=[\mathbf{y}_1,...,\mathbf{y}_N]^T$\f via prior mapping function \(f(\mathbf{x})$\f. The difference to normal GPs is the uncertainty
of \)\mathbf{X}$\f, which will be initialized via PCA and optimized during learning. To avoid memory issues for larger data sets, the algorithm takes use of sparse approximation techniques.
Sparse approximations are used for larger data sets to reduce memory size and computational complexity. This is done by introducing a subset of inducing points or pseudo inputs to approximate the full set. The inversion of the kernel matrix depends only on those points and reduces the computational complexity from \(O(N^3)$\) to $$O(k^2N)$\f, where \(k\) is the number of inducing points and \(N\) the length of the data set.
References:
, 24.11.2019.
Definition at line 66 of file FgAEPSparseDGPSSM.h.
NeuralEngine::MachineLearning::GPModels::AEP::SDGPSSM< Scalar >::SDGPSSM | ( | const af::array & | Y, |
int | latentDimension, | ||
HiddenLayerDescription | description, | ||
Scalar | alpha = 1.0 , |
||
Scalar | priorMean = 0.0 , |
||
Scalar | priorVariance = 1.0 , |
||
af::array | xControl = af::array() , |
||
PropagationMode | probMode = PropagationMode::MomentMatching , |
||
LogLikType | lType = LogLikType::Gaussian , |
||
XInit | emethod = XInit::pca |
||
) |
Constructor.
, 12.06.2018.
Y | The training data. |
X | The training inputs. |
hiddenLayerdescription | The description for one hidden layer. |
alpha | (Optional) The alpha. |
lType | (Optional) likelihood type. |
NeuralEngine::MachineLearning::GPModels::AEP::SDGPSSM< Scalar >::SDGPSSM | ( | const af::array & | Y, |
int | latentDimension, | ||
std::vector< HiddenLayerDescription > | descriptions, | ||
Scalar | alpha = 1.0 , |
||
Scalar | priorMean = 0.0 , |
||
Scalar | priorVariance = 1.0 , |
||
af::array | xControl = af::array() , |
||
PropagationMode | probMode = PropagationMode::MomentMatching , |
||
LogLikType | lType = LogLikType::Gaussian , |
||
XInit | emethod = XInit::pca |
||
) |
Constructor.
, 26.03.2018.
Y | The data af::array to process. |
X | The training inputs. |
hiddenLayerdescriptions | The hidden layer descriptions. |
lType | (Optional) the loglik type. |
NeuralEngine::MachineLearning::GPModels::AEP::SDGPSSM< Scalar >::SDGPSSM | ( | ) |
Default Constructor.
Hmetal T, 29.11.2017.
|
virtual |
Destructor.
, 23.04.2018.
|
overridevirtual |
Cost function the given parameter inputs.
Hmetal T, 29.11.2017.
x | The parameters to be optimized. |
outGradient | [in,out] The out gradient. |
Reimplemented from NeuralEngine::MachineLearning::IModel< Scalar >.
|
protectedvirtual |
Computes the cavity distribution.
Hmetal T, 11/05/2020.
mcav | [in,out] The mcav. |
vcav | [in,out] The vcav. |
cav1 | [in,out] The first cav. |
cav2 | [in,out] The second cav. |
|
protected |
Calculates the tilted transition.
Hmetal T, 11/05/2020.
mprob | The mprob. |
vprob | The vprob. |
mcav_t1 | The first mcav t. |
vcav_t1 | The first vcav t. |
scaleLogZDyn | The scale log z coordinate dynamic. |
dmProb | [in,out] The dm prob. |
dvProb | [in,out] The dv prob. |
dmt | [in,out] The dmt. |
dvt | [in,out] The dvt. |
|
protectedvirtual |
Posterior gradient w.r.t \(\mathbf{X}\).
Hmetal T, 11/05/2020.
|
protectedvirtual |
Cavity gradient w.r.t \(\mathbf{X}\).
Hmetal T, 11/05/2020.
cav1 | The first cav. |
cav2 | The second cav. |
|
protectedvirtual |
LogZ gradient w.r.t \(\mathbf{X}\).
Hmetal T, 11/05/2020.
cav1 | The first cav. |
cav2 | The second cav. |
dmcav_up | The dmcav up. |
dvcav_up | The dvcav up. |
dmcav_prev | The dmcav previous. |
dvcav_prev | The dvcav previous. |
dmcav_next | The dmcav next. |
dvcav_next | The dvcav next. |
|
protectedvirtual |
Calculates the phi prior.
Hmetal T, 11/05/2020.
|
protectedvirtual |
Calculates the phi cavity.
Hmetal T, 11/05/2020.
|
protectedvirtual |
Calculates the phi posterior.
Hmetal T, 11/05/2020.
|
inlineprivate |
Definition at line 232 of file FgAEPSparseDGPSSM.h.
|
friend |
Definition at line 229 of file FgAEPSparseDGPSSM.h.
|
private |
fraction parameter
Definition at line 227 of file FgAEPSparseDGPSSM.h.