Dot product of tensors is not computed in parallel
- To: mathgroup at smc.vnet.net
- Subject: [mg83224] Dot product of tensors is not computed in parallel
- From: chmarkakis at gmail.com
- Date: Thu, 15 Nov 2007 05:42:10 -0500 (EST)
For tensors of rank 2 or lower (matrices and vectors), the Dot command
is executed very efficiently in parallel. Here is an example:
Define two square matrices of real numbers:
In: m1 = Table[Random[Real],{256},{256}];
m2 = Table[Random[Real],{256},{256}];
In a machine with 2 or more CPU cores, compute their dot product (this
machine has 8 cores). The sum of CPU time used in all cores is:
In: Tsingle = Timing[m1.m2][[1]]
Out: 0.010999 Second
The actual time that passed is:
In: Tparallel = AbsoluteTiming[m1.m2][[1]]
Out: 0.001546 Second
For large matrices, the ratio is close to the number of cores (8):
In: Tsingle / Tparallel
Out: 7.114489004
But if we try the same when m1 is a rank 3 tensor, there is no
improvement:
In: m1 = Table[Random[Real],{256},{256},{256}];
m2 = Table[Random[Real],{256},{256}];
In: Tsingle=Timing[m1.m2][[1]]
Out: 8.272743 Second
In: Tparallel=AbsoluteTiming[m1.m2][[1]]
Out: 8.264623 Second
Now the ratio is just 1, i.e. there is no parallelization for tensors!
In: Tsingle / Tparallel
Out: 1.000982501
Is there an efficient way to do the computation of dot products of
tensors (of rank higher than two) in parallel?