ALS optimization for CP and Tucker tensor decompositions
Contents
Alternating least squares for PARAFAC/CANDECOMP
The function parafac_als computes an estimate of the best rank-R PARAFAC model of a tensor X using an alternating least-squares algorithm. The input X can be a tensor, sptensor, ktensor, or ttensor. The result P is a ktensor.
rand('state',0);
X = sptenrand([5 4 3], 10)
X is a sparse tensor of size 5 x 4 x 3 with 10 nonzeros (1,4,1) 0.4966 (2,2,3) 0.8998 (3,2,3) 0.8216 (3,3,1) 0.6449 (3,3,3) 0.8180 (3,4,1) 0.6602 (4,1,2) 0.3420 (4,1,3) 0.2897 (5,2,2) 0.3412 (5,3,2) 0.5341
P = parafac_als(X,2)
CP_ALS:
Iter 1: f = 3.219563e-01 f-delta = 3.2e-01
Iter 2: f = 3.645517e-01 f-delta = 4.3e-02
Iter 3: f = 3.732887e-01 f-delta = 8.7e-03
Iter 4: f = 3.809608e-01 f-delta = 7.7e-03
Iter 5: f = 4.021826e-01 f-delta = 2.1e-02
Iter 6: f = 4.427524e-01 f-delta = 4.1e-02
Iter 7: f = 4.734919e-01 f-delta = 3.1e-02
Iter 8: f = 4.848760e-01 f-delta = 1.1e-02
Iter 9: f = 4.890031e-01 f-delta = 4.1e-03
Iter 10: f = 4.907726e-01 f-delta = 1.8e-03
Iter 11: f = 4.916244e-01 f-delta = 8.5e-04
Iter 12: f = 4.920996e-01 f-delta = 4.8e-04
Iter 13: f = 4.924246e-01 f-delta = 3.2e-04
Iter 14: f = 4.926962e-01 f-delta = 2.7e-04
Iter 15: f = 4.929575e-01 f-delta = 2.6e-04
Iter 16: f = 4.932285e-01 f-delta = 2.7e-04
Iter 17: f = 4.935198e-01 f-delta = 2.9e-04
Iter 18: f = 4.938385e-01 f-delta = 3.2e-04
Iter 19: f = 4.941904e-01 f-delta = 3.5e-04
Iter 20: f = 4.945813e-01 f-delta = 3.9e-04
Iter 21: f = 4.950178e-01 f-delta = 4.4e-04
Iter 22: f = 4.955072e-01 f-delta = 4.9e-04
Iter 23: f = 4.960583e-01 f-delta = 5.5e-04
Iter 24: f = 4.966814e-01 f-delta = 6.2e-04
Iter 25: f = 4.973882e-01 f-delta = 7.1e-04
Iter 26: f = 4.981921e-01 f-delta = 8.0e-04
Iter 27: f = 4.991075e-01 f-delta = 9.2e-04
Iter 28: f = 5.001490e-01 f-delta = 1.0e-03
Iter 29: f = 5.013282e-01 f-delta = 1.2e-03
Iter 30: f = 5.026502e-01 f-delta = 1.3e-03
Iter 31: f = 5.041052e-01 f-delta = 1.5e-03
Iter 32: f = 5.056587e-01 f-delta = 1.6e-03
Iter 33: f = 5.072418e-01 f-delta = 1.6e-03
Iter 34: f = 5.087490e-01 f-delta = 1.5e-03
Iter 35: f = 5.100586e-01 f-delta = 1.3e-03
Iter 36: f = 5.110745e-01 f-delta = 1.0e-03
Iter 37: f = 5.117692e-01 f-delta = 6.9e-04
Iter 38: f = 5.121888e-01 f-delta = 4.2e-04
Iter 39: f = 5.124165e-01 f-delta = 2.3e-04
Iter 40: f = 5.125308e-01 f-delta = 1.1e-04
Iter 41: f = 5.125856e-01 f-delta = 5.5e-05
Final f = 5.125856e-01
P is a ktensor of size 5 x 4 x 3
P.lambda = [ 1.3189 1.1109 ]
P.U{1} =
0.0019 0.2743
0.6406 -0.0177
0.7679 0.9615
-0.0000 0.0000
-0.0000 -0.0000
P.U{2} =
-0.0000 0.0000
0.9413 -0.0855
0.2693 0.7083
-0.2036 0.7007
P.U{3} =
0.0402 0.8828
-0.0000 -0.0000
0.9992 0.4698
P = parafac_als(X,2,struct('dimorder',[3 2 1]))
CP_ALS:
Iter 1: f = 3.575290e-01 f-delta = 3.6e-01
Iter 2: f = 4.968299e-01 f-delta = 1.4e-01
Iter 3: f = 5.047740e-01 f-delta = 7.9e-03
Iter 4: f = 5.084288e-01 f-delta = 3.7e-03
Iter 5: f = 5.103942e-01 f-delta = 2.0e-03
Iter 6: f = 5.114388e-01 f-delta = 1.0e-03
Iter 7: f = 5.119941e-01 f-delta = 5.6e-04
Iter 8: f = 5.122906e-01 f-delta = 3.0e-04
Iter 9: f = 5.124494e-01 f-delta = 1.6e-04
Iter 10: f = 5.125349e-01 f-delta = 8.5e-05
Final f = 5.125349e-01
P is a ktensor of size 5 x 4 x 3
P.lambda = [ 1.3217 1.0933 ]
P.U{1} =
-0.0029 0.2940
0.6361 -0.0293
0.7716 0.9554
0.0000 -0.0000
0.0000 0.0000
P.U{2} =
0.0000 -0.0000
0.9356 -0.0865
0.3018 0.6913
-0.1832 0.7174
P.U{3} =
0.0483 0.9024
0.0000 0.0000
0.9988 0.4308
P = parafac_als(X,2,struct('dimorder',[3 2 1],'init','nvecs'))
CP_ALS:
Iter 1: f = 3.767513e-01 f-delta = 3.8e-01
Iter 2: f = 4.273501e-01 f-delta = 5.1e-02
Iter 3: f = 4.966758e-01 f-delta = 6.9e-02
Iter 4: f = 5.061467e-01 f-delta = 9.5e-03
Iter 5: f = 5.092466e-01 f-delta = 3.1e-03
Iter 6: f = 5.108361e-01 f-delta = 1.6e-03
Iter 7: f = 5.116747e-01 f-delta = 8.4e-04
Iter 8: f = 5.121203e-01 f-delta = 4.5e-04
Iter 9: f = 5.123582e-01 f-delta = 2.4e-04
Iter 10: f = 5.124859e-01 f-delta = 1.3e-04
Iter 11: f = 5.125545e-01 f-delta = 6.9e-05
Final f = 5.125545e-01
P is a ktensor of size 5 x 4 x 3
P.lambda = [ 1.3212 1.0943 ]
P.U{1} =
-0.0028 0.2928
0.6367 -0.0289
0.7711 0.9557
0.0000 -0.0000
0.0000 0.0000
P.U{2} =
0.0000 -0.0000
0.9360 -0.0856
0.2999 0.6927
-0.1842 0.7161
P.U{3} =
0.0471 0.9012
0.0000 0.0000
0.9989 0.4334
U0 = {rand(5,2),rand(4,2),[]}; %<-- Initial guess for factors of P
P = parafac_als(X,2,struct('dimorder',[3 2 1],'init',{U0}))
CP_ALS:
Iter 1: f = 4.361298e-01 f-delta = 4.4e-01
Iter 2: f = 5.082769e-01 f-delta = 7.2e-02
Iter 3: f = 5.105738e-01 f-delta = 2.3e-03
Iter 4: f = 5.116456e-01 f-delta = 1.1e-03
Iter 5: f = 5.121929e-01 f-delta = 5.5e-04
Iter 6: f = 5.124502e-01 f-delta = 2.6e-04
Iter 7: f = 5.125615e-01 f-delta = 1.1e-04
Iter 8: f = 5.126068e-01 f-delta = 4.5e-05
Final f = 5.126068e-01
P is a ktensor of size 5 x 4 x 3
P.lambda = [ 1.3217 1.1037 ]
P.U{1} =
-0.0007 0.2835
0.6381 -0.0241
0.7699 0.9587
0.0000 -0.0000
0.0000 0.0000
P.U{2} =
0.0000 -0.0000
0.9388 -0.0899
0.2834 0.7000
-0.1957 0.7085
P.U{3} =
0.0487 0.8893
0.0000 0.0000
0.9988 0.4573
Alternating least squares for Tucker model
The function tucker_als computes the best rank(R1,R2,..,Rn) approximation of tensor X, according to the specified dimensions in vector R. The input X can be a tensor, sptensor, ktensor, or ttensor. The result returned in T is a ttensor.
X = sptenrand([5 4 3], 10)
X is a sparse tensor of size 5 x 4 x 3 with 10 nonzeros (1,3,1) 0.7400 (3,1,2) 0.4319 (3,2,1) 0.6343 (3,3,2) 0.8030 (4,1,2) 0.0839 (4,2,1) 0.9455 (4,4,2) 0.9159 (4,4,3) 0.6020 (5,3,3) 0.2536 (5,4,3) 0.8735
T = tucker_als(X,2) %<-- best rank(2,2,2) approximation
Tucker Alternating Least-Squares:
Iter 1: fit = 2.810591e-01 fitdelta = 2.8e-01
Iter 2: fit = 3.474829e-01 fitdelta = 6.6e-02
Iter 3: fit = 3.628582e-01 fitdelta = 1.5e-02
Iter 4: fit = 3.700452e-01 fitdelta = 7.2e-03
Iter 5: fit = 3.727897e-01 fitdelta = 2.7e-03
Iter 6: fit = 3.737295e-01 fitdelta = 9.4e-04
Iter 7: fit = 3.740582e-01 fitdelta = 3.3e-04
Iter 8: fit = 3.741751e-01 fitdelta = 1.2e-04
Iter 9: fit = 3.742168e-01 fitdelta = 4.2e-05
T is a ttensor of size 5 x 4 x 3
T.core is a tensor of size 2 x 2 x 2
T.core(:,:,1) =
1.1796 -0.0116
0.4219 -0.0175
T.core(:,:,2) =
0.0098 1.0308
-0.0191 -0.4827
T.U{1} =
0.0069 -0.0204
-0.0000 0.0000
0.2980 -0.6769
0.8904 -0.0567
0.3439 0.7336
T.U{2} =
0.0439 0.0018
0.0204 0.9997
0.1129 0.0109
0.9924 -0.0219
T.U{3} =
0.0109 0.9999
0.6015 -0.0016
0.7988 -0.0124
T = tucker_als(X,[2 2 1]) %<-- best rank(2,2,1) approximation
Tucker Alternating Least-Squares:
Iter 1: fit = 1.812756e-01 fitdelta = 1.8e-01
Iter 2: fit = 2.272937e-01 fitdelta = 4.6e-02
Iter 3: fit = 2.412379e-01 fitdelta = 1.4e-02
Iter 4: fit = 2.436064e-01 fitdelta = 2.4e-03
Iter 5: fit = 2.444688e-01 fitdelta = 8.6e-04
Iter 6: fit = 2.449320e-01 fitdelta = 4.6e-04
Iter 7: fit = 2.451964e-01 fitdelta = 2.6e-04
Iter 8: fit = 2.453474e-01 fitdelta = 1.5e-04
Iter 9: fit = 2.454331e-01 fitdelta = 8.6e-05
T is a ttensor of size 5 x 4 x 3
T.core is a tensor of size 2 x 2 x 1
T.core(:,:,1) =
1.1975 -0.0004
-0.0001 0.7710
T.U{1} =
0.0024 0.0387
0 0
0.0728 0.9885
0.9137 -0.1170
0.3999 0.0872
T.U{2} =
0.0760 0.4549
0.0347 0.0306
0.0869 0.8828
0.9927 -0.1131
T.U{3} =
0.0343
0.8414
0.5394
T = tucker_als(X,2,struct('dimorder',[3 2 1]))
Tucker Alternating Least-Squares:
Iter 1: fit = 3.268831e-01 fitdelta = 3.3e-01
Iter 2: fit = 3.604384e-01 fitdelta = 3.4e-02
Iter 3: fit = 3.708956e-01 fitdelta = 1.0e-02
Iter 4: fit = 3.731357e-01 fitdelta = 2.2e-03
Iter 5: fit = 3.738515e-01 fitdelta = 7.2e-04
Iter 6: fit = 3.741016e-01 fitdelta = 2.5e-04
Iter 7: fit = 3.741906e-01 fitdelta = 8.9e-05
T is a ttensor of size 5 x 4 x 3
T.core is a tensor of size 2 x 2 x 2
T.core(:,:,1) =
1.1797 -0.0054
0.4208 -0.0338
T.core(:,:,2) =
0.0015 1.0306
-0.0375 -0.4818
T.U{1} =
0.0069 -0.0208
0.0000 -0.0000
0.2981 -0.6769
0.8904 -0.0566
0.3439 0.7336
T.U{2} =
0.0440 0.0028
0.0323 0.9992
0.1134 0.0181
0.9921 -0.0347
T.U{3} =
0.0298 0.9994
0.6017 -0.0051
0.7982 -0.0335
T = tucker_als(X,2,struct('dimorder',[3 2 1],'init','eigs'))
Computing 2 leading e-vectors for factor 2.
Computing 2 leading e-vectors for factor 1.
Tucker Alternating Least-Squares:
Iter 1: fit = 3.726300e-01 fitdelta = 3.7e-01
Iter 2: fit = 3.741337e-01 fitdelta = 1.5e-03
Iter 3: fit = 3.742335e-01 fitdelta = 1.0e-04
T is a ttensor of size 5 x 4 x 3
T.core is a tensor of size 2 x 2 x 2
T.core(:,:,1) =
1.1798 -0.0000
0.4220 -0.0000
T.core(:,:,2) =
0.0000 1.0311
-0.0000 -0.4828
T.U{1} =
0.0000 -0.0000
0.0000 -0.0000
0.2970 -0.6795
0.8913 -0.0548
0.3426 0.7316
T.U{2} =
0.0427 0
0.0000 1.0000
0.1082 0.0000
0.9932 -0.0000
T.U{3} =
0.0000 1.0000
0.6045 0
0.7966 -0.0000
U0 = {rand(5,2),rand(4,2),[]}; %<-- Initial guess for factors of T
T = tucker_als(X,2,struct('dimorder',[3 2 1],'init',{U0}))
Tucker Alternating Least-Squares:
Iter 1: fit = 3.647914e-01 fitdelta = 3.6e-01
Iter 2: fit = 3.722524e-01 fitdelta = 7.5e-03
Iter 3: fit = 3.735753e-01 fitdelta = 1.3e-03
Iter 4: fit = 3.740042e-01 fitdelta = 4.3e-04
Iter 5: fit = 3.741559e-01 fitdelta = 1.5e-04
Iter 6: fit = 3.742100e-01 fitdelta = 5.4e-05
T is a ttensor of size 5 x 4 x 3
T.core is a tensor of size 2 x 2 x 2
T.core(:,:,1) =
1.1797 -0.0042
0.4214 -0.0265
T.core(:,:,2) =
0.0012 1.0308
-0.0293 -0.4823
T.U{1} =
0.0054 -0.0162
0.0000 0
0.2980 -0.6769
0.8904 -0.0567
0.3439 0.7337
T.U{2} =
0.0440 0.0022
0.0253 0.9995
0.1131 0.0141
0.9923 -0.0272
T.U{3} =
0.0233 0.9997
0.6016 -0.0040
0.7985 -0.0262