Sparse GP layer. More...
#include <FgAEPSparseGPLayer.h>


Public Member Functions | |
| SGPLayer (int numPoints, int numPseudos, int outputDim, int inputDim) | |
| Constructor. More... | |
| virtual | ~SGPLayer () |
| Destructor. More... | |
| void | ForwardPredictionCavity (af::array &outMout, af::array &outVout, af::array *psi1out, af::array *psi2out, const af::array &mx, const af::array *vx=nullptr, Scalar alpha=1.0, PropagationMode mode=PropagationMode::MomentMatching) |
| Forward prediction through cavity. More... | |
| void | ForwardPredictionRandomCavityMC (af::array &mout, af::array &vout, af::array &xout, af::array &eps, const af::array &mx, const af::array &vx, Scalar alpha=1.0) |
| Forward prediction random cavity MCMC. More... | |
| af::array | BackpropGradientsReg (const af::array &m, const af::array &v, const af::array &dlogZ_dm, const af::array &dlogZ_dv, const af::array &x, std::map< std::string, af::array > *outGrad_cav=nullptr, Scalar alpha=1.0) |
| Back propagation gradients through usual GP regression task. More... | |
| af::array | BackpropGradientsMM (const af::array &m, const af::array &v, const af::array &dlogZ_dm, const af::array &dlogZ_dv, const af::array &psi1, const af::array &psi2, const af::array &mx, const af::array &vx, std::map< std::string, af::array > *outGrad_cav, Scalar alpha=1.0) |
| Back propagation gradients through moment matching. More... | |
| af::array | BackpropGradientsMC (const af::array &m, const af::array &v, const af::array &eps, const af::array &dlogZ_dm, const af::array &dlogZ_dv, const af::array &x, std::map< std::string, af::array > *outGradInput, Scalar alpha=1.0) |
| Back propagation gradients through MCMC. More... | |
| Scalar | ComputePhi (Scalar alpha) |
| Computes the weighted sum of the log-partitions of prior, post and cav. More... | |
| void | UpdateFactor (af::array &n, std::map< std::string, af::array > grad_cav, Scalar alpha, Scalar decay=0) |
| Update step of PowerEP. More... | |
| virtual void | UpdateParameters () override |
| Updates the parameters. More... | |
Public Member Functions inherited from NeuralEngine::MachineLearning::GPModels::SparseGPBaseLayer< Scalar > | |
| SparseGPBaseLayer (int numPoints, int numPseudos, int outputDim, int inputDim) | |
| Constructor. More... | |
| virtual | ~SparseGPBaseLayer () |
| Destructor. More... | |
| virtual void | ForwardPredictionPost (const af::array *mx, const af::array *vx, af::array &mout, af::array &vout) override |
| Forward prediction of posterior function values. More... | |
| virtual void | SampleFromPost (const af::array &inX, af::array &outfsample) override |
| Samples from posterior. More... | |
| void | ComputeKuu () |
| Calculates the kernel matrix of pseudo inputs. More... | |
| void | ComputeKfu (const af::array &inX) |
| Calculates the kernel matrix of inputes and pseudo inputs. More... | |
| af::array | GetPseudoInputs () |
| Gets pseudo inputs. More... | |
| virtual int | GetNumParameters () override |
| Gets number of parameters to be optimized. More... | |
| virtual void | SetParameters (const af::array ¶m) override |
| Sets the parameters for each optimization iteration. More... | |
| virtual af::array | GetParameters () override |
| Gets the parameters for each optimization iteration. More... | |
| virtual void | FixInducing (bool isfixed) |
| Set fixation for inducing inputs. More... | |
| virtual void | UpdateParameters () override |
| Updates the parameters. More... | |
| virtual void | InitParameters (af::array *X=nullptr) override |
| Initializes the parameters. More... | |
Public Member Functions inherited from NeuralEngine::MachineLearning::GPModels::GPBaseLayer< Scalar > | |
| GPBaseLayer (int numPoints, int outputDim, int inputDim) | |
| Constructor. More... | |
| virtual | ~GPBaseLayer () |
| Destructor. More... | |
| IKernel< Scalar > * | GetKernel () |
| Gets the kernel function. More... | |
| void | SetKernel (IKernel< Scalar > *kern) |
| Sets a kernel function. More... | |
| virtual void | InitParameters (af::array *X=nullptr) |
| virtual int | GetNumParameters () |
| Gets number of parameters to be optimized. More... | |
| virtual void | SetParameters (const af::array ¶m) |
| Sets the parameters for each optimization iteration. More... | |
| virtual af::array | GetParameters () |
| Gets the parameters for each optimization iteration. More... | |
| virtual void | ForwardPredictionPost (const af::array *mx, const af::array *vx, af::array &mout, af::array &vout) |
| Forward prediction of posterior function values. More... | |
| virtual void | SampleFromPost (const af::array &inX, af::array &outfsample) |
| Samples from posterior. More... | |
| virtual void | FixKernelParameters (bool isfixed) |
| Sets fixation for hyperparameters. More... | |
| virtual void | UpdateParameters () |
| Updates the parameters. More... | |
| virtual void | SetDataSize (int length, int dimension) override |
| Sets data size. More... | |
| void | SetStyles (std::map< std::string, Style< Scalar > > *styles) |
| Sets the syles. More... | |
| void | SetLatentDimension (int q) |
| Sets latent dimension. More... | |
Public Member Functions inherited from NeuralEngine::MachineLearning::ILayer< Scalar > | |
| ILayer (LayerType type, int numPoints, int outputDim) | |
| Constructor. More... | |
| virtual | ~ILayer ()=default |
| Destructor. More... | |
| LayerType | GetType () |
| Gets the layer type. More... | |
| virtual int | GetNumParameters ()=0 |
| Gets number of parameters to be optimized. More... | |
| virtual void | SetParameters (const af::array ¶m)=0 |
| Sets the parameters for each optimization iteration. More... | |
| virtual af::array | GetParameters ()=0 |
| Gets the parameters for each optimization iteration. More... | |
| virtual void | UpdateParameters ()=0 |
| Updates the parameters. More... | |
| virtual void | SetDataSize (int length, int dimension) |
| Sets data size. More... | |
Protected Member Functions | |
| SGPLayer () | |
| Default constructor. More... | |
| void | ForwardPredictionDeterministicPost (const af::array &mx, af::array *mout, af::array *vout) override |
| Deterministic forward propagation through posterior. More... | |
| void | ForwardPredictionDeterministicCavity (af::array &outMout, af::array &outVout, af::array *kfuOut, const af::array &mx, Scalar alpha=1.0) |
| Forward prediction through deterministic cavity. More... | |
| void | ForwardPredictionRandomCavity (af::array &mout, af::array &vout, af::array *psi1out, af::array *psi2out, const af::array &mx, const af::array &vx, PropagationMode mode, Scalar alpha=1.0) |
| Forward prediction through random cavity. More... | |
| void | ForwardPredictionRandomCavityMM (af::array &mout, af::array &vout, af::array *psi1out, af::array *psi2out, const af::array &mx, const af::array &vx, Scalar alpha=1.0) |
| Forward prediction random cavity moment matching. More... | |
| virtual void | ForwardPredictionRandomPost (const af::array &mx, const af::array &vx, af::array &mout, af::array &vout, PropagationMode mode=PropagationMode::MomentMatching) override |
| Forward prediction through random posterior. More... | |
| void | ForwardPredictionRandomPostMM (const af::array &mx, const af::array &vx, af::array &mout, af::array &vout) |
| Forward prediction through posterior via Moment Matching. More... | |
| void | ComputeCavity (Scalar alpha=1.0f) |
| Calculates the cavity. More... | |
| void | ComputeCavityGradientU (af::array &dMucav, af::array &dSucav, af::array &out_dT1, af::array &out_dT2, af::array &out_dInvKuu, Scalar alpha=1.0f) |
| Calculates gradient contributions of cavity distribution. More... | |
| void | ComputePosteriorGradientU (af::array &dMu, af::array &dSu, af::array &out_dT1, af::array &out_dT2, af::array &out_dInvKuu) |
| Calculates the gradient contributions of posterior. More... | |
| Scalar | ComputePhiPrior () |
| Calculates energy contribution phi prior. More... | |
| Scalar | ComputePhiPosterior () |
| Calculates energy contribution phi posterior. More... | |
| Scalar | ComputePhiCavity () |
| Calculates energy contribution phi cavity. More... | |
Protected Member Functions inherited from NeuralEngine::MachineLearning::GPModels::SparseGPBaseLayer< Scalar > | |
| SparseGPBaseLayer () | |
| Default constructor. More... | |
| virtual void | ReinitParameters () override |
| Reinitializes the parameters. More... | |
| virtual void | ForwardPredictionDeterministicPost (const af::array &mx, af::array *mout, af::array *vout) |
| Deterministic forward propagation through posterior. More... | |
| virtual void | ForwardPredictionRandomPost (const af::array &mx, const af::array &vx, af::array &mout, af::array &vout, PropagationMode mode=PropagationMode::MomentMatching) |
| Forward prediction through random posterior. More... | |
Protected Member Functions inherited from NeuralEngine::MachineLearning::GPModels::GPBaseLayer< Scalar > | |
| GPBaseLayer () | |
| Default constructor. More... | |
| virtual void | ReinitParameters () |
| Reinitializes the parameters. More... | |
Protected Member Functions inherited from NeuralEngine::MachineLearning::ILayer< Scalar > | |
| ILayer () | |
| Default constructor. More... | |
Protected Attributes | |
| af::array | afGamma |
| af::array | afBeta |
| af::array | afGammaHat |
| af::array | afBetaHat |
| af::array | afSuMuMu |
| af::array | afBetaStochastic |
| af::array | afBetaHatStochastic |
| af::array | afMuHat |
| af::array | afSuHat |
| af::array | afInvSuHat |
| af::array | afInvSuMuHat |
| af::array | afSuMuMuHat |
Protected Attributes inherited from NeuralEngine::MachineLearning::GPModels::SparseGPBaseLayer< Scalar > | |
| int | ik |
| af::array | afMu |
| af::array | afSu |
| af::array | afInvSu |
| af::array | afInvSuMu |
| af::array | T1 |
| af::array | T2 |
| af::array | T2_R |
| af::array | afXu |
| af::array | afKuu |
| af::array | afInvKuu |
| af::array | afKfu |
| bool | isFixedInducing |
Protected Attributes inherited from NeuralEngine::MachineLearning::GPModels::GPBaseLayer< Scalar > | |
| int | iq |
| Latent dimension. More... | |
| bool | isFixedHypers |
| IKernel< Scalar > * | kernel |
| kernel function More... | |
| std::map< std::string, Style< Scalar > > * | mStyles |
| style variable More... | |
| Scalar | JITTER |
| for kernel matrix stability (positive definiteness) More... | |
Protected Attributes inherited from NeuralEngine::MachineLearning::ILayer< Scalar > | |
| int | iD |
| data dimension More... | |
| int | iN |
| data size More... | |
| LayerType | lType |
| liklihood or gp layer More... | |
| af::dtype | m_dType |
| floating point precision flag for af::array More... | |
Private Member Functions | |
| template<class Archive > | |
| void | serialize (Archive &ar, unsigned int version) |
Friends | |
| class | boost::serialization::access |
| template<typename > | |
| class | SDGPR |
Sparse GP layer.
Holds all variables for FITC approximation and PEP. Defines a subset of \(\mathbf{X}\). Selects \(k\) inducing inputs \(\mathbf{X_u}\), computes the subset kernel matrix \(\mathbf{K_{uu}}\) and its inverse.
For more information see, Hmetal T, 05/05/2018.
Definition at line 39 of file FgAEPSparseGPLayer.h.
| NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::SGPLayer | ( | int | numPoints, |
| int | numPseudos, | ||
| int | outputDim, | ||
| int | inputDim | ||
| ) |
Constructor.
, 15.05.2018.
| numPoints | Number of points. |
| numPseudos | Number of pseudo inputs. |
| outputDim | The output dimension. |
| inputDim | The input dimension. |
|
virtual |
Destructor.
, 15.05.2018.
|
inlineprotected |
| void NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::ForwardPredictionCavity | ( | af::array & | outMout, |
| af::array & | outVout, | ||
| af::array * | psi1out, | ||
| af::array * | psi2out, | ||
| const af::array & | mx, | ||
| const af::array * | vx = nullptr, |
||
| Scalar | alpha = 1.0, |
||
| PropagationMode | mode = PropagationMode::MomentMatching |
||
| ) |
Forward prediction through cavity.
Computes new mean m^{
}_{f} and covariance V^{
}_{ff} function. From cavity distribution.
, 16.05.2018.
| mout | [in,out] The m^{ }_{f}. |
| vout | [in,out] The V^{ }_{ff}. |
| mx | [in,out] The inputs mx. |
| vx | [in,out] (Optional) If non-null, the variances vx. |
| alpha | (Optional) the alpha, weighning for alpha-divergence. |
| void NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::ForwardPredictionRandomCavityMC | ( | af::array & | mout, |
| af::array & | vout, | ||
| af::array & | xout, | ||
| af::array & | eps, | ||
| const af::array & | mx, | ||
| const af::array & | vx, | ||
| Scalar | alpha = 1.0 |
||
| ) |
Forward prediction random cavity MCMC.
Hmetal T, 10/07/2019.
| mout | [in,out] The mout. |
| vout | [in,out] The vout. |
| mx | The mx. |
| vx | The vx. |
| alpha | (Optional) the alpha. |
| af::array NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::BackpropGradientsReg | ( | const af::array & | m, |
| const af::array & | v, | ||
| const af::array & | dlogZ_dm, | ||
| const af::array & | dlogZ_dv, | ||
| const af::array & | x, | ||
| std::map< std::string, af::array > * | outGrad_cav = nullptr, |
||
| Scalar | alpha = 1.0 |
||
| ) |
Back propagation gradients through usual GP regression task.
Hmetal T, 11/07/2019.
| outGrad_hyper | [in,out] Gradient of hypers. |
| m | Cavity prediction mean from current layer. |
| v | Cavity prediction variance from current layer. |
| dlogZ_dm | Cavity prediction mean gradient. |
| dlogZ_dv | Cavity prediction variance gradient. |
| x | The training inputs. |
| outGrad_cav | (Optional) [in,out] Gradient of cavity parameters. |
| alpha | (Optional) Ratio. |
| af::array NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::BackpropGradientsMM | ( | const af::array & | m, |
| const af::array & | v, | ||
| const af::array & | dlogZ_dm, | ||
| const af::array & | dlogZ_dv, | ||
| const af::array & | psi1, | ||
| const af::array & | psi2, | ||
| const af::array & | mx, | ||
| const af::array & | vx, | ||
| std::map< std::string, af::array > * | outGrad_cav, | ||
| Scalar | alpha = 1.0 |
||
| ) |
Back propagation gradients through moment matching.
Hmetal T, 11/07/2019.
| m | Cavity prediction mean from current layer. |
| v | Cavity prediction variance from current layer. |
| dlogZ_dm | Cavity prediction mean gradient. |
| dlogZ_dv | Cavity prediction variance gradient. |
| psi1 | Cavity Psi1 statistics. |
| psi2 | Cavity Psi2 statistics. |
| mx | Cavity prediction mean from lower layer. |
| vx | Cavity prediction variance from lower layer. |
| outGrad_hyper | [in,out] Gradient of hypers. |
| outGrad_cav | [in,out] Gradient of cavity parameters. |
| alpha | (Optional) Ratio. |
| af::array NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::BackpropGradientsMC | ( | const af::array & | m, |
| const af::array & | v, | ||
| const af::array & | eps, | ||
| const af::array & | dlogZ_dm, | ||
| const af::array & | dlogZ_dv, | ||
| const af::array & | x, | ||
| std::map< std::string, af::array > * | outGradInput, | ||
| Scalar | alpha = 1.0 |
||
| ) |
Back propagation gradients through MCMC.
Hmetal T, 11/07/2019.
| m | Cavity prediction mean from current layer. |
| v | Cavity prediction variance from current layer. |
| dlogZ_dm | Cavity prediction mean gradient. |
| dlogZ_dv | Cavity prediction variance gradient. |
| x | The training inputs. |
| outGrad_cav | [in,out] (Optional) Gradient of cavity parameters. |
| alpha | (Optional) Ratio. |
| Scalar NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::ComputePhi | ( | Scalar | alpha | ) |
Computes the weighted sum of the log-partitions of prior, post and cav.
, 25.06.2018.
| alpha | The alpha. |
| void NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::UpdateFactor | ( | af::array & | n, |
| std::map< std::string, af::array > | grad_cav, | ||
| Scalar | alpha, | ||
| Scalar | decay = 0 |
||
| ) |
Update step of PowerEP.
Hmetal T, 08/06/2018.
| n | [in,out] The indexes to compute. |
| grad_cav | [in,out] The gradient of the cavity functions m_f^{ } and V_{ff}^{\m}. |
| alpha | The alpha. |
| decay | (Optional) the decay. |
|
overridevirtual |
Updates the parameters.
, 26.06.2018.
Reimplemented from NeuralEngine::MachineLearning::GPModels::SparseGPBaseLayer< Scalar >.
|
overrideprotectedvirtual |
Deterministic forward propagation through posterior.
Hmetal T, 01.04.2019.
| mx | [in,out] The inputs mx. |
| mout | [in,out] The m^{ }_{f}. |
| vout | [in,out] The V^{ }_{ff}. |
Reimplemented from NeuralEngine::MachineLearning::GPModels::SparseGPBaseLayer< Scalar >.
|
protected |
Forward prediction through deterministic cavity.
Projection step. Computation of cavity posterior mean and covariance function,
$$m^{
}_{f} = K_{fu}K_{uu}^{−1}T^{
, −1}_{2, u}T^{
}_{1, u},$$
$$V{
}_{ff} = K_{ff} − Q_{ff} + K_{fu}K^{−1}_{uu}T^{
, −1}_{2, u}K^{−1}_{uu}K_{uf}.$$
, 16.05.2018.
| mout | [in,out] The m^{ }_{f}. |
| vout | [in,out] The V^{ }_{ff}. |
| idx | [in,out] indexes of points to be removed from distribution. |
| mx | [in,out] The inputs mx. |
| alpha | (Optional) the alpha, weighning for alpha-divergence. |
|
protected |
Forward prediction through random cavity.
, 16.05.2018.
| idx | [in,out] indexes of points to be removed from distribution. |
| mout | [in,out] The m^{ }_{f}. |
| vout | [in,out] The V^{ }_{ff}. |
| mx | The inputs mx. |
| vx | If non-null, the variances vx. |
| mode | Propagation mode. |
| parameter6 | (Optional) the alpha, ratio for alpha-divergence. |
|
protected |
Forward prediction random cavity moment matching.
Hmetal T, 10/07/2019.
| mout | [in,out] The mout. |
| vout | [in,out] The vout. |
| mx | The mx. |
| vx | The vx. |
| alpha | (Optional) the alpha. |
|
overrideprotectedvirtual |
Forward prediction through random posterior.
, 16.05.2018.
| mx | The inputs mx. |
| vx | If non-null, the variances vx. |
| mout | [in,out] The m^{ }_{f}. |
| vout | [in,out] The V^{ }_{ff}. |
| mode | (Optional) Propagation mode. |
Reimplemented from NeuralEngine::MachineLearning::GPModels::SparseGPBaseLayer< Scalar >.
|
protected |
Forward prediction through posterior via Moment Matching.
Hmetal T, 17/09/2019.
| mx | The inputs mx. |
| vx | The input variances vx. |
| mout | [in,out] The mout. |
| vout | [in,out] The vout. |
|
protected |
Calculates the cavity.
Deletion step: The cavity for data point n, q^{
}(f) ∝ q^∗(f)/t^α_n(u), has a similar form to the posterior, but the natural parameters are modified by deletion,
T^\n_{1, u} = T_{1, u} − αT_{1, n}
and
T^\n_{2, u} = T_{2, u} − αT_{2, n},
for yielding new mean \^{\mathbf{m_u}} and covariance function \^{\mathbf{S_u} for cavity distribution.
, 16.05.2018.
| alpha | (Optional) the alpha, weighning for alpha-divergence. |
|
protected |
Calculates gradient contributions of cavity distribution.
, 22.06.2018.
| dMucav | [in,out] The gradient mucav. |
| dSucav | [in,out] The gradient sucav. |
| out_dT1 | [in,out] The gradient of natural parameter 1. |
| out_dT2 | [in,out] The gradient of natural parameter 2. |
| out_dInvKuu | [in,out] The gradient of inverse Kuu. |
| alpha | (Optional) the alpha. |
|
protected |
Calculates the gradient contributions of posterior.
, 25.06.2018.
| dMu | [in,out] The gradient mu. |
| dSu | [in,out] The gradient su. |
| out_dT1 | [in,out] The gradient of natural parameter 1. |
| out_dT2 | [in,out] The gradient of natural parameter 2. |
|
protected |
Calculates energy contribution phi prior.
, 26.06.2018.
|
protected |
Calculates energy contribution phi posterior.
, 26.06.2018.
|
protected |
Calculates energy contribution phi cavity.
, 26.06.2018.
|
inlineprivate |
Definition at line 366 of file FgAEPSparseGPLayer.h.
|
friend |
Definition at line 362 of file FgAEPSparseGPLayer.h.
Definition at line 363 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 346 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 347 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 348 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 349 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 350 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 351 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 352 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 355 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 356 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 357 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 358 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 359 of file FgAEPSparseGPLayer.h.