Algorithms for computing tensor decompositions

Contents

Alternating least squares for PARAFAC/CANDECOMP

The function parafac_als computes an estimate of the best rank-R PARAFAC model of a tensor X using an alternating least-squares algorithm. The input X can be a tensor, sptensor, ktensor, or ttensor. The result P is a ktensor.

rand('state',0);
X = sptenrand([5 4 3], 10)
X is a sparse tensor of size 5 x 4 x 3 with 10 nonzeros
	(1,4,1)    0.4966
	(2,2,3)    0.8998
	(3,2,3)    0.8216
	(3,3,1)    0.6449
	(3,3,3)    0.8180
	(3,4,1)    0.6602
	(4,1,2)    0.3420
	(4,1,3)    0.2897
	(5,2,2)    0.3412
	(5,3,2)    0.5341
P = parafac_als(X,2)
CP_ALS:
 Iter  1: fit = 3.219563e-001 fitdelta = 3.2e-001
 Iter  2: fit = 3.645517e-001 fitdelta = 4.3e-002
 Iter  3: fit = 3.732887e-001 fitdelta = 8.7e-003
 Iter  4: fit = 3.809608e-001 fitdelta = 7.7e-003
 Iter  5: fit = 4.021826e-001 fitdelta = 2.1e-002
 Iter  6: fit = 4.427524e-001 fitdelta = 4.1e-002
 Iter  7: fit = 4.734919e-001 fitdelta = 3.1e-002
 Iter  8: fit = 4.848760e-001 fitdelta = 1.1e-002
 Iter  9: fit = 4.890031e-001 fitdelta = 4.1e-003
 Iter 10: fit = 4.907726e-001 fitdelta = 1.8e-003
 Iter 11: fit = 4.916244e-001 fitdelta = 8.5e-004
 Iter 12: fit = 4.920996e-001 fitdelta = 4.8e-004
 Iter 13: fit = 4.924246e-001 fitdelta = 3.2e-004
 Iter 14: fit = 4.926962e-001 fitdelta = 2.7e-004
 Iter 15: fit = 4.929575e-001 fitdelta = 2.6e-004
 Iter 16: fit = 4.932285e-001 fitdelta = 2.7e-004
 Iter 17: fit = 4.935198e-001 fitdelta = 2.9e-004
 Iter 18: fit = 4.938385e-001 fitdelta = 3.2e-004
 Iter 19: fit = 4.941904e-001 fitdelta = 3.5e-004
 Iter 20: fit = 4.945813e-001 fitdelta = 3.9e-004
 Iter 21: fit = 4.950178e-001 fitdelta = 4.4e-004
 Iter 22: fit = 4.955072e-001 fitdelta = 4.9e-004
 Iter 23: fit = 4.960583e-001 fitdelta = 5.5e-004
 Iter 24: fit = 4.966814e-001 fitdelta = 6.2e-004
 Iter 25: fit = 4.973882e-001 fitdelta = 7.1e-004
 Iter 26: fit = 4.981921e-001 fitdelta = 8.0e-004
 Iter 27: fit = 4.991075e-001 fitdelta = 9.2e-004
 Iter 28: fit = 5.001490e-001 fitdelta = 1.0e-003
 Iter 29: fit = 5.013282e-001 fitdelta = 1.2e-003
 Iter 30: fit = 5.026502e-001 fitdelta = 1.3e-003
 Iter 31: fit = 5.041052e-001 fitdelta = 1.5e-003
 Iter 32: fit = 5.056587e-001 fitdelta = 1.6e-003
 Iter 33: fit = 5.072418e-001 fitdelta = 1.6e-003
 Iter 34: fit = 5.087490e-001 fitdelta = 1.5e-003
 Iter 35: fit = 5.100586e-001 fitdelta = 1.3e-003
 Iter 36: fit = 5.110745e-001 fitdelta = 1.0e-003
 Iter 37: fit = 5.117692e-001 fitdelta = 6.9e-004
 Iter 38: fit = 5.121888e-001 fitdelta = 4.2e-004
 Iter 39: fit = 5.124165e-001 fitdelta = 2.3e-004
 Iter 40: fit = 5.125308e-001 fitdelta = 1.1e-004
 Iter 41: fit = 5.125856e-001 fitdelta = 5.5e-005
 Final fit = 5.125856e-001 
P is a ktensor of size 5 x 4 x 3
	P.lambda = [ 1.3189      1.1109 ]
	P.U{1} = 
		    0.0019    0.2743
		    0.6406   -0.0177
		    0.7679    0.9615
		   -0.0000    0.0000
		   -0.0000   -0.0000
	P.U{2} = 
		   -0.0000    0.0000
		    0.9413   -0.0855
		    0.2693    0.7083
		   -0.2036    0.7007
	P.U{3} = 
		    0.0402    0.8828
		   -0.0000   -0.0000
		    0.9992    0.4698
P = parafac_als(X,2,struct('dimorder',[3 2 1]))
CP_ALS:
 Iter  1: fit = 3.575290e-001 fitdelta = 3.6e-001
 Iter  2: fit = 4.968299e-001 fitdelta = 1.4e-001
 Iter  3: fit = 5.047740e-001 fitdelta = 7.9e-003
 Iter  4: fit = 5.084288e-001 fitdelta = 3.7e-003
 Iter  5: fit = 5.103942e-001 fitdelta = 2.0e-003
 Iter  6: fit = 5.114388e-001 fitdelta = 1.0e-003
 Iter  7: fit = 5.119941e-001 fitdelta = 5.6e-004
 Iter  8: fit = 5.122906e-001 fitdelta = 3.0e-004
 Iter  9: fit = 5.124494e-001 fitdelta = 1.6e-004
 Iter 10: fit = 5.125349e-001 fitdelta = 8.5e-005
 Final fit = 5.125349e-001 
P is a ktensor of size 5 x 4 x 3
	P.lambda = [ 1.3217      1.0933 ]
	P.U{1} = 
		   -0.0029    0.2940
		    0.6361   -0.0293
		    0.7716    0.9554
		    0.0000   -0.0000
		    0.0000    0.0000
	P.U{2} = 
		    0.0000   -0.0000
		    0.9356   -0.0865
		    0.3018    0.6913
		   -0.1832    0.7174
	P.U{3} = 
		    0.0483    0.9024
		    0.0000    0.0000
		    0.9988    0.4308
P = parafac_als(X,2,struct('dimorder',[3 2 1],'init','nvecs'))
CP_ALS:
 Iter  1: fit = 3.767513e-001 fitdelta = 3.8e-001
 Iter  2: fit = 4.273501e-001 fitdelta = 5.1e-002
 Iter  3: fit = 4.966758e-001 fitdelta = 6.9e-002
 Iter  4: fit = 5.061467e-001 fitdelta = 9.5e-003
 Iter  5: fit = 5.092466e-001 fitdelta = 3.1e-003
 Iter  6: fit = 5.108361e-001 fitdelta = 1.6e-003
 Iter  7: fit = 5.116747e-001 fitdelta = 8.4e-004
 Iter  8: fit = 5.121203e-001 fitdelta = 4.5e-004
 Iter  9: fit = 5.123582e-001 fitdelta = 2.4e-004
 Iter 10: fit = 5.124859e-001 fitdelta = 1.3e-004
 Iter 11: fit = 5.125545e-001 fitdelta = 6.9e-005
 Final fit = 5.125545e-001 
P is a ktensor of size 5 x 4 x 3
	P.lambda = [ 1.3212      1.0943 ]
	P.U{1} = 
		   -0.0028    0.2928
		    0.6367   -0.0289
		    0.7711    0.9557
		    0.0000   -0.0000
		    0.0000    0.0000
	P.U{2} = 
		    0.0000   -0.0000
		    0.9360   -0.0856
		    0.2999    0.6927
		   -0.1842    0.7161
	P.U{3} = 
		    0.0471    0.9012
		    0.0000    0.0000
		    0.9989    0.4334
U0 = {rand(5,2),rand(4,2),[]}; %<-- Initial guess for factors of P
P = parafac_als(X,2,struct('dimorder',[3 2 1],'init',{U0}))
CP_ALS:
 Iter  1: fit = 4.361298e-001 fitdelta = 4.4e-001
 Iter  2: fit = 5.082769e-001 fitdelta = 7.2e-002
 Iter  3: fit = 5.105738e-001 fitdelta = 2.3e-003
 Iter  4: fit = 5.116456e-001 fitdelta = 1.1e-003
 Iter  5: fit = 5.121929e-001 fitdelta = 5.5e-004
 Iter  6: fit = 5.124502e-001 fitdelta = 2.6e-004
 Iter  7: fit = 5.125615e-001 fitdelta = 1.1e-004
 Iter  8: fit = 5.126068e-001 fitdelta = 4.5e-005
 Final fit = 5.126068e-001 
P is a ktensor of size 5 x 4 x 3
	P.lambda = [ 1.3217      1.1037 ]
	P.U{1} = 
		   -0.0007    0.2835
		    0.6381   -0.0241
		    0.7699    0.9587
		    0.0000   -0.0000
		    0.0000    0.0000
	P.U{2} = 
		    0.0000   -0.0000
		    0.9388   -0.0899
		    0.2834    0.7000
		   -0.1957    0.7085
	P.U{3} = 
		    0.0487    0.8893
		    0.0000    0.0000
		    0.9988    0.4573

Alternating least squares for Tucker model

The function tucker_als computes the best rank(R1,R2,..,Rn) approximation of tensor X, according to the specified dimensions in vector R. The input X can be a tensor, sptensor, ktensor, or ttensor. The result returned in T is a ttensor.

X = sptenrand([5 4 3], 10)
X is a sparse tensor of size 5 x 4 x 3 with 10 nonzeros
	(1,3,1)    0.7400
	(3,1,2)    0.4319
	(3,2,1)    0.6343
	(3,3,2)    0.8030
	(4,1,2)    0.0839
	(4,2,1)    0.9455
	(4,4,2)    0.9159
	(4,4,3)    0.6020
	(5,3,3)    0.2536
	(5,4,3)    0.8735
T = tucker_als(X,2)        %<-- best rank(2,2,2) approximation
Tucker Alternating Least-Squares:
 Iter  1: fit = 2.810591e-001 fitdelta = 2.8e-001
 Iter  2: fit = 3.474829e-001 fitdelta = 6.6e-002
 Iter  3: fit = 3.628582e-001 fitdelta = 1.5e-002
 Iter  4: fit = 3.700452e-001 fitdelta = 7.2e-003
 Iter  5: fit = 3.727897e-001 fitdelta = 2.7e-003
 Iter  6: fit = 3.737295e-001 fitdelta = 9.4e-004
 Iter  7: fit = 3.740582e-001 fitdelta = 3.3e-004
 Iter  8: fit = 3.741751e-001 fitdelta = 1.2e-004
 Iter  9: fit = 3.742168e-001 fitdelta = 4.2e-005
T is a ttensor of size 5 x 4 x 3
	T.core is a tensor of size 2 x 2 x 2
		T.core(:,:,1) = 
	    1.1796   -0.0116
	    0.4219   -0.0175
		T.core(:,:,2) = 
	    0.0098    1.0308
	   -0.0191   -0.4827
	T.U{1} = 
		    0.0069   -0.0204
		   -0.0000    0.0000
		    0.2980   -0.6769
		    0.8904   -0.0567
		    0.3439    0.7336
	T.U{2} = 
		    0.0439    0.0018
		    0.0204    0.9997
		    0.1129    0.0109
		    0.9924   -0.0219
	T.U{3} = 
		    0.0109    0.9999
		    0.6015   -0.0016
		    0.7988   -0.0124
T = tucker_als(X,[2 2 1])  %<-- best rank(2,2,1) approximation
Tucker Alternating Least-Squares:
 Iter  1: fit = 1.812756e-001 fitdelta = 1.8e-001
 Iter  2: fit = 2.272937e-001 fitdelta = 4.6e-002
 Iter  3: fit = 2.412379e-001 fitdelta = 1.4e-002
 Iter  4: fit = 2.436064e-001 fitdelta = 2.4e-003
 Iter  5: fit = 2.444688e-001 fitdelta = 8.6e-004
 Iter  6: fit = 2.449320e-001 fitdelta = 4.6e-004
 Iter  7: fit = 2.451964e-001 fitdelta = 2.6e-004
 Iter  8: fit = 2.453474e-001 fitdelta = 1.5e-004
 Iter  9: fit = 2.454331e-001 fitdelta = 8.6e-005
T is a ttensor of size 5 x 4 x 3
	T.core is a tensor of size 2 x 2 x 1
		T.core(:,:,1) = 
	    1.1975   -0.0004
	   -0.0001    0.7710
	T.U{1} = 
		    0.0024    0.0387
		    0.0000    0.0000
		    0.0728    0.9885
		    0.9137   -0.1170
		    0.3999    0.0872
	T.U{2} = 
		    0.0760    0.4549
		    0.0347    0.0306
		    0.0869    0.8828
		    0.9927   -0.1131
	T.U{3} = 
		    0.0343
		    0.8414
		    0.5394
T = tucker_als(X,2,struct('dimorder',[3 2 1]))
Tucker Alternating Least-Squares:
 Iter  1: fit = 3.268831e-001 fitdelta = 3.3e-001
 Iter  2: fit = 3.604384e-001 fitdelta = 3.4e-002
 Iter  3: fit = 3.708956e-001 fitdelta = 1.0e-002
 Iter  4: fit = 3.731357e-001 fitdelta = 2.2e-003
 Iter  5: fit = 3.738515e-001 fitdelta = 7.2e-004
 Iter  6: fit = 3.741016e-001 fitdelta = 2.5e-004
 Iter  7: fit = 3.741906e-001 fitdelta = 8.9e-005
T is a ttensor of size 5 x 4 x 3
	T.core is a tensor of size 2 x 2 x 2
		T.core(:,:,1) = 
	    1.1797   -0.0054
	    0.4208   -0.0338
		T.core(:,:,2) = 
	    0.0015    1.0306
	   -0.0375   -0.4818
	T.U{1} = 
		    0.0069   -0.0208
		         0         0
		    0.2981   -0.6769
		    0.8904   -0.0566
		    0.3439    0.7336
	T.U{2} = 
		    0.0440    0.0028
		    0.0323    0.9992
		    0.1134    0.0181
		    0.9921   -0.0347
	T.U{3} = 
		    0.0298    0.9994
		    0.6017   -0.0051
		    0.7982   -0.0335
T = tucker_als(X,2,struct('dimorder',[3 2 1],'init','eigs'))
  Computing 2 leading e-vectors for factor 2.
  Computing 2 leading e-vectors for factor 1.

Tucker Alternating Least-Squares:
 Iter  1: fit = 3.726300e-001 fitdelta = 3.7e-001
 Iter  2: fit = 3.741337e-001 fitdelta = 1.5e-003
 Iter  3: fit = 3.742335e-001 fitdelta = 1.0e-004
T is a ttensor of size 5 x 4 x 3
	T.core is a tensor of size 2 x 2 x 2
		T.core(:,:,1) = 
	    1.1798    0.0000
	    0.4220   -0.0000
		T.core(:,:,2) = 
	   -0.0000    1.0311
	   -0.0000   -0.4828
	T.U{1} = 
		    0.0000         0
		         0    0.0000
		    0.2970   -0.6795
		    0.8913   -0.0548
		    0.3426    0.7316
	T.U{2} = 
		    0.0427    0.0000
		   -0.0000    1.0000
		    0.1082    0.0000
		    0.9932    0.0000
	T.U{3} = 
		    0.0000    1.0000
		    0.6045   -0.0000
		    0.7966   -0.0000
U0 = {rand(5,2),rand(4,2),[]}; %<-- Initial guess for factors of T
T = tucker_als(X,2,struct('dimorder',[3 2 1],'init',{U0}))
Tucker Alternating Least-Squares:
 Iter  1: fit = 3.647914e-001 fitdelta = 3.6e-001
 Iter  2: fit = 3.722524e-001 fitdelta = 7.5e-003
 Iter  3: fit = 3.735753e-001 fitdelta = 1.3e-003
 Iter  4: fit = 3.740042e-001 fitdelta = 4.3e-004
 Iter  5: fit = 3.741559e-001 fitdelta = 1.5e-004
 Iter  6: fit = 3.742100e-001 fitdelta = 5.4e-005
T is a ttensor of size 5 x 4 x 3
	T.core is a tensor of size 2 x 2 x 2
		T.core(:,:,1) = 
	    1.1797   -0.0042
	    0.4214   -0.0265
		T.core(:,:,2) = 
	    0.0012    1.0308
	   -0.0293   -0.4823
	T.U{1} = 
		    0.0054   -0.0162
		    0.0000         0
		    0.2980   -0.6769
		    0.8904   -0.0567
		    0.3439    0.7337
	T.U{2} = 
		    0.0440    0.0022
		    0.0253    0.9995
		    0.1131    0.0141
		    0.9923   -0.0272
	T.U{3} = 
		    0.0233    0.9997
		    0.6016   -0.0040
		    0.7985   -0.0262