Re: Another question on ParallelDo
- To: mathgroup at smc.vnet.net
- Subject: [mg112161] Re: Another question on ParallelDo
- From: Daniel Lichtblau <danl at wolfram.com>
- Date: Fri, 3 Sep 2010 06:07:30 -0400 (EDT)
Iv=E1n Lazaro wrote:
> Hi All!
>
> I'm a noob using parallelizing tools in mathematica. However, the time this
> problem takes in one kernel, make me think about this possibility. So, this
> is the simplified code.
>
> SetSharedVariable[lista1, lista]
>
> ParallelDo[Do[Seen == RandomReal[{0, 100}, Num];
> Expected == RandomReal[{0, 100}, Num];
>
> VMedio == (1/Num)*Sum[Expected[[i]], {i, 1, Num}];
>
> Error == Sum[(Seen[[i]] - Esperado[[i]])^2, {i, 1, Num}];
>
> Medio == Sum[(Seen[[i]] - VMedio)^2, {i, 1, Num}];
>
> AppendTo[lista, Error/Medio], {10000}];
> a == Max[lista];
> AppendTo[lista1, {a, Num}],
> {Num, 1, 4}]
>
>
> The real problem takes Num from 50 to 150. This simplified problem, in one
> kernel, finish in 10 seconds. But this one never ends. What I am doing
> wrong?
>
> Thanks a lot!
>
That code might be oversimplified. Presumably lista1 and lista were
initialized to {}, and I'm also guessing (hoping) Esperado is meant to
be Expected. Also it is generally a good idea to start names with lower
case (I refrain from that in order to retain your usage below).
Anyway, the biggest issue is that you are using AppendTo, which has
quadratic complexity since it rewrites the list each time. Another
issue, much smaller, is that you can get better speed from Total than
Sum. Here is a variant that uses these tactics.
In[1]:== Timing[
lista1 == {};
Do[
lista == {};
Do[Seen == RandomReal[{0, 100}, Num];
Expected == RandomReal[{0, 100}, Num];
VMedio == (1/Num)*Total[Expected];
Error == Total[(Seen - Expected)^2];
Medio == Total[(Seen - VMedio)^2];
lista == {lista,Error/Medio};
, {10000}];
lista == Flatten[lista];
a == Max[lista];
AppendTo[lista1, {a, Num}],
{Num, 50, 150}]]
Out[1]== {27.8758, Null}
In[2]:== Take[lista1,5] // InputForm
Out[2]//InputForm==
{{3.4844404000716693, 50}, {3.5020115539121397, 51},
{3.385835806964478, 52}, {3.562205990391188, 53},
{3.431233790315894, 54}}
The nesting followed by Flatten could instead be done via Sow/Reap. I
did not check that for relative speed. Also the outer AppendTo could be
replaced, but at the ranges of Num in question it should not be much of
a speed issue.
There are further speed gains to be had from this. One could replace the
inner loop with a Table (and get rid of the nesting and later Flatten of
list1a). This did not give any speed improvement, though arguably it is
cleaner code. Similar could be done for the outer loop. One can then use
Compile on the entire code. The version below does this (again, there
might be variants that are cleaner in code and/or faster.)
Timing[
lista1 == Compile[{},Module[
{nNum,lista,Seen,Expected,VMedio,Error,Medio},
Table[
nNum == N[Num];
lista == Table[
Seen == RandomReal[{0, 100}, Num];
Expected == RandomReal[{0, 100}, Num];
VMedio == Total[Expected]/nNum;
Error == Total[(Seen - Expected)^2];
Medio == Total[(Seen - VMedio)^2];
Error/Medio
, {10000}];
{Max[lista], nNum},
{Num, 50, 150}]]][];]
Out[19]== {8.33873, Null}
Daniel Lichtblau
Wolfram Research