Re: Parallel Kit Question: ParallelDot is much more slow than Dot
- To: mathgroup at smc.vnet.net
- Subject: [mg40434] Re: Parallel Kit Question: ParallelDot is much more slow than Dot
- From: "Gareth J. Russell" <gjr2008 at columbia.edu>
- Date: Sat, 5 Apr 2003 03:59:52 -0500 (EST)
- Organization: Columbia University
- References: <b6gn64$ckk$1@smc.vnet.net> <b6j8ev$5vc$1@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
True, but in case you put people off ever using a parallel set-up, let me point out where it is useful and easy: simulations and related programming. If I want to run the same simulation 1000 times, returning 1000 values, and that takes an hour, I can distribute the task to ten machines that do 100 each, and that will take just a little over 6 minutes. And all by putting the function that represents the simulation into ParallelTable. Gareth Russell Columbia University In <b6j8ev$5vc$1 at smc.vnet.net> Jens-Peer Kuska wrote: > Hi, > > parallel commands are usualy slower than serial ones, > because you have the overhead for process communication. > > For your parallel dot command you have to transfer two > doubles (16 Byte) to do a multiplication and a addition > with double speed. I don't know you CPU type and speed. > I would expect, that you CPU can do > an addition and a multiplication in 2-4 CPU-cylces > and this will need much less time than the transfer > of 16 Byte via MathLink. > Since the time for sending the data is much larger than > the speed gain by the parallel execution, your parallel > command must be much slower than the serial version. > > General it is very difficult to design an > algorithm that is faster executed parallel than on a > serial machine. The critical question is "how fast can the > comunicatione be" and "how can I reduce the data exchange" > because a communication via shared memory, pipes or TCP/IP > is typical 100-10000 slower than the data exchange on the > system bus on the mainboard. > > To get a speed gain, you have to increase the operation > count that every CPU does on the data. It must > be much lager than a simple addition/multiplication. > Say 5000-10000 CPU cycles per byte - than and only than you will > see a speed gain. > > Don't expect, that any of your parallel commands > is faster than the serial execution. It is a > fairy-tale that parallel computing is faster > than serial execution. The exceptions of this > rule need special algorithms and very carefull > programming. > > Regards > Jens > > > > Denis Areshkin wrote: >> >> Dear All, >> >> I appreciate any response on the following problem: >> >> It appears that ParrallelDot function, which is a >> part of Parallel Kit, works two orders of magnitude >> slower on my two processor machine than Dot. Below >> is the commands used to evaluate dot product in pa- >> rallel. I am not sure that the command >> >> ExportEnvironment[TestMtrx]; >> >> is necessary in that case (though it doesn't do any >> harm). It takes 6 seconds to evaluate >> >> Dot[TestMtrx,TestMtrx] >> >> and about half an hour to evaluate >> >> ParallelDot[TestMtrx,TestMtrx] >> >> Needs["Parallel`Parallel`"] >> >> Needs["Parallel`Commands`"] >> >> ProcIDTable = Table[LaunchSlave["localhost",$mathkernel],{2}]; >> >> TestMtrx = Table[Random[Real,{-1,1}],{1000},{1000}]; >> >> ExportEnvironment[TestMtrx]; >> >> TestDot = ParallelDot[TestMtrx,TestMtrx] >> >> I have plenty of RAM (4GB), thus this problem can't >> be caused by kernels competing for memory space... >> >> Thank you for your help >> >> -- >> >> Denis Areshkin >> (919) 513-2424 (office) >> (919) 835-1650 (home) > >