Sparse GP layer. More...
#include <FgAEPSparseGPLayer.h>
Public Member Functions | |
SGPLayer (int numPoints, int numPseudos, int outputDim, int inputDim) | |
Constructor. More... | |
virtual | ~SGPLayer () |
Destructor. More... | |
void | ForwardPredictionCavity (af::array &outMout, af::array &outVout, af::array *psi1out, af::array *psi2out, const af::array &mx, const af::array *vx=nullptr, Scalar alpha=1.0, PropagationMode mode=PropagationMode::MomentMatching) |
Forward prediction through cavity. More... | |
void | ForwardPredictionRandomCavityMC (af::array &mout, af::array &vout, af::array &xout, af::array &eps, const af::array &mx, const af::array &vx, Scalar alpha=1.0) |
Forward prediction random cavity MCMC. More... | |
af::array | BackpropGradientsReg (const af::array &m, const af::array &v, const af::array &dlogZ_dm, const af::array &dlogZ_dv, const af::array &x, std::map< std::string, af::array > *outGrad_cav=nullptr, Scalar alpha=1.0) |
Back propagation gradients through usual GP regression task. More... | |
af::array | BackpropGradientsMM (const af::array &m, const af::array &v, const af::array &dlogZ_dm, const af::array &dlogZ_dv, const af::array &psi1, const af::array &psi2, const af::array &mx, const af::array &vx, std::map< std::string, af::array > *outGrad_cav, Scalar alpha=1.0) |
Back propagation gradients through moment matching. More... | |
af::array | BackpropGradientsMC (const af::array &m, const af::array &v, const af::array &eps, const af::array &dlogZ_dm, const af::array &dlogZ_dv, const af::array &x, std::map< std::string, af::array > *outGradInput, Scalar alpha=1.0) |
Back propagation gradients through MCMC. More... | |
Scalar | ComputePhi (Scalar alpha) |
Computes the weighted sum of the log-partitions of prior, post and cav. More... | |
void | UpdateFactor (af::array &n, std::map< std::string, af::array > grad_cav, Scalar alpha, Scalar decay=0) |
Update step of PowerEP. More... | |
virtual void | UpdateParameters () override |
Updates the parameters. More... | |
![]() | |
SparseGPBaseLayer (int numPoints, int numPseudos, int outputDim, int inputDim) | |
Constructor. More... | |
virtual | ~SparseGPBaseLayer () |
Destructor. More... | |
virtual void | ForwardPredictionPost (const af::array *mx, const af::array *vx, af::array &mout, af::array &vout) override |
Forward prediction of posterior function values. More... | |
virtual void | SampleFromPost (const af::array &inX, af::array &outfsample) override |
Samples from posterior. More... | |
void | ComputeKuu () |
Calculates the kernel matrix of pseudo inputs. More... | |
void | ComputeKfu (const af::array &inX) |
Calculates the kernel matrix of inputes and pseudo inputs. More... | |
af::array | GetPseudoInputs () |
Gets pseudo inputs. More... | |
virtual int | GetNumParameters () override |
Gets number of parameters to be optimized. More... | |
virtual void | SetParameters (const af::array ¶m) override |
Sets the parameters for each optimization iteration. More... | |
virtual af::array | GetParameters () override |
Gets the parameters for each optimization iteration. More... | |
virtual void | FixInducing (bool isfixed) |
Set fixation for inducing inputs. More... | |
virtual void | UpdateParameters () override |
Updates the parameters. More... | |
virtual void | InitParameters (af::array *X=nullptr) override |
Initializes the parameters. More... | |
![]() | |
GPBaseLayer (int numPoints, int outputDim, int inputDim) | |
Constructor. More... | |
virtual | ~GPBaseLayer () |
Destructor. More... | |
IKernel< Scalar > * | GetKernel () |
Gets the kernel function. More... | |
void | SetKernel (IKernel< Scalar > *kern) |
Sets a kernel function. More... | |
virtual void | InitParameters (af::array *X=nullptr) |
virtual int | GetNumParameters () |
Gets number of parameters to be optimized. More... | |
virtual void | SetParameters (const af::array ¶m) |
Sets the parameters for each optimization iteration. More... | |
virtual af::array | GetParameters () |
Gets the parameters for each optimization iteration. More... | |
virtual void | ForwardPredictionPost (const af::array *mx, const af::array *vx, af::array &mout, af::array &vout) |
Forward prediction of posterior function values. More... | |
virtual void | SampleFromPost (const af::array &inX, af::array &outfsample) |
Samples from posterior. More... | |
virtual void | FixKernelParameters (bool isfixed) |
Sets fixation for hyperparameters. More... | |
virtual void | UpdateParameters () |
Updates the parameters. More... | |
virtual void | SetDataSize (int length, int dimension) override |
Sets data size. More... | |
void | SetStyles (std::map< std::string, Style< Scalar > > *styles) |
Sets the syles. More... | |
void | SetLatentDimension (int q) |
Sets latent dimension. More... | |
![]() | |
ILayer (LayerType type, int numPoints, int outputDim) | |
Constructor. More... | |
virtual | ~ILayer ()=default |
Destructor. More... | |
LayerType | GetType () |
Gets the layer type. More... | |
virtual int | GetNumParameters ()=0 |
Gets number of parameters to be optimized. More... | |
virtual void | SetParameters (const af::array ¶m)=0 |
Sets the parameters for each optimization iteration. More... | |
virtual af::array | GetParameters ()=0 |
Gets the parameters for each optimization iteration. More... | |
virtual void | UpdateParameters ()=0 |
Updates the parameters. More... | |
virtual void | SetDataSize (int length, int dimension) |
Sets data size. More... | |
Protected Member Functions | |
SGPLayer () | |
Default constructor. More... | |
void | ForwardPredictionDeterministicPost (const af::array &mx, af::array *mout, af::array *vout) override |
Deterministic forward propagation through posterior. More... | |
void | ForwardPredictionDeterministicCavity (af::array &outMout, af::array &outVout, af::array *kfuOut, const af::array &mx, Scalar alpha=1.0) |
Forward prediction through deterministic cavity. More... | |
void | ForwardPredictionRandomCavity (af::array &mout, af::array &vout, af::array *psi1out, af::array *psi2out, const af::array &mx, const af::array &vx, PropagationMode mode, Scalar alpha=1.0) |
Forward prediction through random cavity. More... | |
void | ForwardPredictionRandomCavityMM (af::array &mout, af::array &vout, af::array *psi1out, af::array *psi2out, const af::array &mx, const af::array &vx, Scalar alpha=1.0) |
Forward prediction random cavity moment matching. More... | |
virtual void | ForwardPredictionRandomPost (const af::array &mx, const af::array &vx, af::array &mout, af::array &vout, PropagationMode mode=PropagationMode::MomentMatching) override |
Forward prediction through random posterior. More... | |
void | ForwardPredictionRandomPostMM (const af::array &mx, const af::array &vx, af::array &mout, af::array &vout) |
Forward prediction through posterior via Moment Matching. More... | |
void | ComputeCavity (Scalar alpha=1.0f) |
Calculates the cavity. More... | |
void | ComputeCavityGradientU (af::array &dMucav, af::array &dSucav, af::array &out_dT1, af::array &out_dT2, af::array &out_dInvKuu, Scalar alpha=1.0f) |
Calculates gradient contributions of cavity distribution. More... | |
void | ComputePosteriorGradientU (af::array &dMu, af::array &dSu, af::array &out_dT1, af::array &out_dT2, af::array &out_dInvKuu) |
Calculates the gradient contributions of posterior. More... | |
Scalar | ComputePhiPrior () |
Calculates energy contribution phi prior. More... | |
Scalar | ComputePhiPosterior () |
Calculates energy contribution phi posterior. More... | |
Scalar | ComputePhiCavity () |
Calculates energy contribution phi cavity. More... | |
![]() | |
SparseGPBaseLayer () | |
Default constructor. More... | |
virtual void | ReinitParameters () override |
Reinitializes the parameters. More... | |
virtual void | ForwardPredictionDeterministicPost (const af::array &mx, af::array *mout, af::array *vout) |
Deterministic forward propagation through posterior. More... | |
virtual void | ForwardPredictionRandomPost (const af::array &mx, const af::array &vx, af::array &mout, af::array &vout, PropagationMode mode=PropagationMode::MomentMatching) |
Forward prediction through random posterior. More... | |
![]() | |
GPBaseLayer () | |
Default constructor. More... | |
virtual void | ReinitParameters () |
Reinitializes the parameters. More... | |
![]() | |
ILayer () | |
Default constructor. More... | |
Protected Attributes | |
af::array | afGamma |
af::array | afBeta |
af::array | afGammaHat |
af::array | afBetaHat |
af::array | afSuMuMu |
af::array | afBetaStochastic |
af::array | afBetaHatStochastic |
af::array | afMuHat |
af::array | afSuHat |
af::array | afInvSuHat |
af::array | afInvSuMuHat |
af::array | afSuMuMuHat |
![]() | |
int | ik |
af::array | afMu |
af::array | afSu |
af::array | afInvSu |
af::array | afInvSuMu |
af::array | T1 |
af::array | T2 |
af::array | T2_R |
af::array | afXu |
af::array | afKuu |
af::array | afInvKuu |
af::array | afKfu |
bool | isFixedInducing |
![]() | |
int | iq |
Latent dimension. More... | |
bool | isFixedHypers |
IKernel< Scalar > * | kernel |
kernel function More... | |
std::map< std::string, Style< Scalar > > * | mStyles |
style variable More... | |
Scalar | JITTER |
for kernel matrix stability (positive definiteness) More... | |
![]() | |
int | iD |
data dimension More... | |
int | iN |
data size More... | |
LayerType | lType |
liklihood or gp layer More... | |
af::dtype | m_dType |
floating point precision flag for af::array More... | |
Private Member Functions | |
template<class Archive > | |
void | serialize (Archive &ar, unsigned int version) |
Friends | |
class | boost::serialization::access |
template<typename > | |
class | SDGPR |
Sparse GP layer.
Holds all variables for FITC approximation and PEP. Defines a subset of \(\mathbf{X}\). Selects \(k\) inducing inputs \(\mathbf{X_u}\), computes the subset kernel matrix \(\mathbf{K_{uu}}\) and its inverse.
For more information see, Hmetal T, 05/05/2018.
Definition at line 39 of file FgAEPSparseGPLayer.h.
NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::SGPLayer | ( | int | numPoints, |
int | numPseudos, | ||
int | outputDim, | ||
int | inputDim | ||
) |
Constructor.
, 15.05.2018.
numPoints | Number of points. |
numPseudos | Number of pseudo inputs. |
outputDim | The output dimension. |
inputDim | The input dimension. |
|
virtual |
Destructor.
, 15.05.2018.
|
inlineprotected |
void NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::ForwardPredictionCavity | ( | af::array & | outMout, |
af::array & | outVout, | ||
af::array * | psi1out, | ||
af::array * | psi2out, | ||
const af::array & | mx, | ||
const af::array * | vx = nullptr , |
||
Scalar | alpha = 1.0 , |
||
PropagationMode | mode = PropagationMode::MomentMatching |
||
) |
Forward prediction through cavity.
Computes new mean m^{
}_{f} and covariance V^{
}_{ff} function. From cavity distribution.
, 16.05.2018.
mout | [in,out] The m^{ }_{f}. |
vout | [in,out] The V^{ }_{ff}. |
mx | [in,out] The inputs mx. |
vx | [in,out] (Optional) If non-null, the variances vx. |
alpha | (Optional) the alpha, weighning for alpha-divergence. |
void NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::ForwardPredictionRandomCavityMC | ( | af::array & | mout, |
af::array & | vout, | ||
af::array & | xout, | ||
af::array & | eps, | ||
const af::array & | mx, | ||
const af::array & | vx, | ||
Scalar | alpha = 1.0 |
||
) |
Forward prediction random cavity MCMC.
Hmetal T, 10/07/2019.
mout | [in,out] The mout. |
vout | [in,out] The vout. |
mx | The mx. |
vx | The vx. |
alpha | (Optional) the alpha. |
af::array NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::BackpropGradientsReg | ( | const af::array & | m, |
const af::array & | v, | ||
const af::array & | dlogZ_dm, | ||
const af::array & | dlogZ_dv, | ||
const af::array & | x, | ||
std::map< std::string, af::array > * | outGrad_cav = nullptr , |
||
Scalar | alpha = 1.0 |
||
) |
Back propagation gradients through usual GP regression task.
Hmetal T, 11/07/2019.
outGrad_hyper | [in,out] Gradient of hypers. |
m | Cavity prediction mean from current layer. |
v | Cavity prediction variance from current layer. |
dlogZ_dm | Cavity prediction mean gradient. |
dlogZ_dv | Cavity prediction variance gradient. |
x | The training inputs. |
outGrad_cav | (Optional) [in,out] Gradient of cavity parameters. |
alpha | (Optional) Ratio. |
af::array NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::BackpropGradientsMM | ( | const af::array & | m, |
const af::array & | v, | ||
const af::array & | dlogZ_dm, | ||
const af::array & | dlogZ_dv, | ||
const af::array & | psi1, | ||
const af::array & | psi2, | ||
const af::array & | mx, | ||
const af::array & | vx, | ||
std::map< std::string, af::array > * | outGrad_cav, | ||
Scalar | alpha = 1.0 |
||
) |
Back propagation gradients through moment matching.
Hmetal T, 11/07/2019.
m | Cavity prediction mean from current layer. |
v | Cavity prediction variance from current layer. |
dlogZ_dm | Cavity prediction mean gradient. |
dlogZ_dv | Cavity prediction variance gradient. |
psi1 | Cavity Psi1 statistics. |
psi2 | Cavity Psi2 statistics. |
mx | Cavity prediction mean from lower layer. |
vx | Cavity prediction variance from lower layer. |
outGrad_hyper | [in,out] Gradient of hypers. |
outGrad_cav | [in,out] Gradient of cavity parameters. |
alpha | (Optional) Ratio. |
af::array NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::BackpropGradientsMC | ( | const af::array & | m, |
const af::array & | v, | ||
const af::array & | eps, | ||
const af::array & | dlogZ_dm, | ||
const af::array & | dlogZ_dv, | ||
const af::array & | x, | ||
std::map< std::string, af::array > * | outGradInput, | ||
Scalar | alpha = 1.0 |
||
) |
Back propagation gradients through MCMC.
Hmetal T, 11/07/2019.
m | Cavity prediction mean from current layer. |
v | Cavity prediction variance from current layer. |
dlogZ_dm | Cavity prediction mean gradient. |
dlogZ_dv | Cavity prediction variance gradient. |
x | The training inputs. |
outGrad_cav | [in,out] (Optional) Gradient of cavity parameters. |
alpha | (Optional) Ratio. |
Scalar NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::ComputePhi | ( | Scalar | alpha | ) |
Computes the weighted sum of the log-partitions of prior, post and cav.
, 25.06.2018.
alpha | The alpha. |
void NeuralEngine::MachineLearning::GPModels::AEP::SGPLayer< Scalar >::UpdateFactor | ( | af::array & | n, |
std::map< std::string, af::array > | grad_cav, | ||
Scalar | alpha, | ||
Scalar | decay = 0 |
||
) |
Update step of PowerEP.
Hmetal T, 08/06/2018.
n | [in,out] The indexes to compute. |
grad_cav | [in,out] The gradient of the cavity functions m_f^{ } and V_{ff}^{\m}. |
alpha | The alpha. |
decay | (Optional) the decay. |
|
overridevirtual |
Updates the parameters.
, 26.06.2018.
Reimplemented from NeuralEngine::MachineLearning::GPModels::SparseGPBaseLayer< Scalar >.
|
overrideprotectedvirtual |
Deterministic forward propagation through posterior.
Hmetal T, 01.04.2019.
mx | [in,out] The inputs mx. |
mout | [in,out] The m^{ }_{f}. |
vout | [in,out] The V^{ }_{ff}. |
Reimplemented from NeuralEngine::MachineLearning::GPModels::SparseGPBaseLayer< Scalar >.
|
protected |
Forward prediction through deterministic cavity.
Projection step. Computation of cavity posterior mean and covariance function,
$$m^{
}_{f} = K_{fu}K_{uu}^{−1}T^{
, −1}_{2, u}T^{
}_{1, u},$$
$$V{
}_{ff} = K_{ff} − Q_{ff} + K_{fu}K^{−1}_{uu}T^{
, −1}_{2, u}K^{−1}_{uu}K_{uf}.$$
, 16.05.2018.
mout | [in,out] The m^{ }_{f}. |
vout | [in,out] The V^{ }_{ff}. |
idx | [in,out] indexes of points to be removed from distribution. |
mx | [in,out] The inputs mx. |
alpha | (Optional) the alpha, weighning for alpha-divergence. |
|
protected |
Forward prediction through random cavity.
, 16.05.2018.
idx | [in,out] indexes of points to be removed from distribution. |
mout | [in,out] The m^{ }_{f}. |
vout | [in,out] The V^{ }_{ff}. |
mx | The inputs mx. |
vx | If non-null, the variances vx. |
mode | Propagation mode. |
parameter6 | (Optional) the alpha, ratio for alpha-divergence. |
|
protected |
Forward prediction random cavity moment matching.
Hmetal T, 10/07/2019.
mout | [in,out] The mout. |
vout | [in,out] The vout. |
mx | The mx. |
vx | The vx. |
alpha | (Optional) the alpha. |
|
overrideprotectedvirtual |
Forward prediction through random posterior.
, 16.05.2018.
mx | The inputs mx. |
vx | If non-null, the variances vx. |
mout | [in,out] The m^{ }_{f}. |
vout | [in,out] The V^{ }_{ff}. |
mode | (Optional) Propagation mode. |
Reimplemented from NeuralEngine::MachineLearning::GPModels::SparseGPBaseLayer< Scalar >.
|
protected |
Forward prediction through posterior via Moment Matching.
Hmetal T, 17/09/2019.
mx | The inputs mx. |
vx | The input variances vx. |
mout | [in,out] The mout. |
vout | [in,out] The vout. |
|
protected |
Calculates the cavity.
Deletion step: The cavity for data point n, q^{
}(f) ∝ q^∗(f)/t^α_n(u), has a similar form to the posterior, but the natural parameters are modified by deletion,
T^\n_{1, u} = T_{1, u} − αT_{1, n} and T^\n_{2, u} = T_{2, u} − αT_{2, n},
for yielding new mean \^{\mathbf{m_u}} and covariance function \^{\mathbf{S_u} for cavity distribution.
, 16.05.2018.
alpha | (Optional) the alpha, weighning for alpha-divergence. |
|
protected |
Calculates gradient contributions of cavity distribution.
, 22.06.2018.
dMucav | [in,out] The gradient mucav. |
dSucav | [in,out] The gradient sucav. |
out_dT1 | [in,out] The gradient of natural parameter 1. |
out_dT2 | [in,out] The gradient of natural parameter 2. |
out_dInvKuu | [in,out] The gradient of inverse Kuu. |
alpha | (Optional) the alpha. |
|
protected |
Calculates the gradient contributions of posterior.
, 25.06.2018.
dMu | [in,out] The gradient mu. |
dSu | [in,out] The gradient su. |
out_dT1 | [in,out] The gradient of natural parameter 1. |
out_dT2 | [in,out] The gradient of natural parameter 2. |
|
protected |
Calculates energy contribution phi prior.
, 26.06.2018.
|
protected |
Calculates energy contribution phi posterior.
, 26.06.2018.
|
protected |
Calculates energy contribution phi cavity.
, 26.06.2018.
|
inlineprivate |
Definition at line 366 of file FgAEPSparseGPLayer.h.
|
friend |
Definition at line 362 of file FgAEPSparseGPLayer.h.
Definition at line 363 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 346 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 347 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 348 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 349 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 350 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 351 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 352 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 355 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 356 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 357 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 358 of file FgAEPSparseGPLayer.h.
|
protected |
Definition at line 359 of file FgAEPSparseGPLayer.h.