Mathematica 9 is now available
Services & Resources / Wolfram Forums
-----
 /
MathGroup Archive
2002
*January
*February
*March
*April
*May
*June
*July
*August
*September
*October
*November
*December
*Archive Index
*Ask about this page
*Print this page
*Give us feedback
*Sign up for the Wolfram Insider

MathGroup Archive 2002

[Date Index] [Thread Index] [Author Index]

Search the Archive

RE: Re: Particular structure 2

  • To: mathgroup at smc.vnet.net
  • Subject: [mg33657] RE: [mg33627] Re: Particular structure 2
  • From: "Wolf, Hartmut" <Hartmut.Wolf at t-systems.com>
  • Date: Thu, 4 Apr 2002 19:40:43 -0500 (EST)
  • Sender: owner-wri-mathgroup at wolfram.com

Please see at bottom of message

> -----Original Message-----
> From: Allan Hayes [mailto:hay at haystack.demon.co.uk]
To: mathgroup at smc.vnet.net
> Sent: Thursday, April 04, 2002 1:09 AM
> To: mathgroup at smc.vnet.net
> Subject: [mg33657] [mg33627] Re: Particular structure 2
> 
> 
> Yas emailed me privately asking for a less memory demanding 
> soluton than my
> earlier on using Outer.
> Here is my reply
> 
> How about this (Quit needs to be evaluated each time 
> separately - or you
> could quit the kernel from the kernel menu)
> 
> New code
> 
>     Quit
> 
>     x= y=Table[1,{400}];f=Plus;F= Times@@#&;
> 
>     Table[Function[kk,ReleaseHold[#]][y[[i]]],{i,Length[y]}]& [
>       Hold[F][f[#,kk,z]&/@x]];//Timing
> 
>             {15.21 Second,Null}
> 
>     MaxMemoryUsed[ ]
> 
>             1385656
> 
> Old code
> 
>     Quit
> 
>     x= y=Table[1,{400}];f=Plus;F= Times@@#&;
> 
>     F/@Map[First,Transpose[Outer[f,x,y,{z}]],{2}];//Timing
> 
>             {29.11 Second,Null}
> 
>     MaxMemoryUsed[ ]
> 
>                     19250032
> Compare to     1385656
> 
> 
> 
> 
> --
> Allan
> 
> ---------------------
> Allan Hayes
> Mathematica Training and Consulting
> Leicester UK
> www.haystack.demon.co.uk
> hay at haystack.demon.co.uk
> Voice: +44 (0)116 271 4198
> Fax: +44 (0)870 164 0565
> 
> 
> "Allan Hayes" <hay at haystack.demon.co.uk> wrote in message
> news:a86k5e$jm4$1 at smc.vnet.net...
> > Yas,
> >
> > F/@Map[First,Transpose[Outer[f,{x1,x2,x3},{y1,y2,y3,y4},{z}]],{2}]
> >
> >
> {F[{f[x1,y1,z],f[x2,y1,z],f[x3,y1,z]}],F[{f[x1,y2,z],f[x2,y2,z
> ],f[x3,y2,z]}]
> > ,
> >
> >
> F[{f[x1,y3,z],f[x2,y3,z],f[x3,y3,z]}],F[{f[x1,y4,z],f[x2,y4,z]
> ,f[x3,y4,z]}]}
> >
> > Where F is the function to be used at the second stage.
> >
> > --
> > Allan
> >
> > ---------------------
> > Allan Hayes
> > Mathematica Training and Consulting
> > Leicester UK
> > www.haystack.demon.co.uk
> > hay at haystack.demon.co.uk
> > Voice: +44 (0)116 271 4198
> > Fax: +44 (0)870 164 0565
> >
> >
> > "Yas" <yast at optushome.com.au> wrote in message
> > news:a81jgj$778$1 at smc.vnet.net...
> > > G'day,
> > > How do I go about achieving the structured list below,
> > >
> > > {f[x1, y1, z], ... , f[xn, y1, z]}
> > >
> > > followed by some operation on the above list (e.g Fold), store the
> > > result and then, do the same again for other y values, as in,
> > >
> > > {f[x1, ym, z], ... , f[xn, ym, z]}
> > >
> > > until the last value of ym.
> > >
> > > In summary, the n values of x get slotted in first for 
> one value of y
> > > then the resulting list is evaluated, the answer stored, 
> values of x get
> > > slotted in again, y is incremented and so forth until all 
> (m) values of
> > > y have been done.
> > >
> > > Thanks
> > > Yas
> > >
> > >
> >
> >
> >
> 
> 
> 

Dear Allan,

I was let to ponder for a while why you held F in the Map over x. Without
knowing more details of the problem, a "best solution" can hardly be
proposed. Of course there might be cases where the evaluation of F might go
astray unless the values y[[i]] are inserted. But normally, in the interest
of memory consumption and performance (partial) evaluation should be done as
early as possible. 


Here are some of my observations:

(1) reproducing your case above

In[1]:=
x = y = Table[1, {400}]; f = Plus; g = Times @@ # &; z = 1;

In[2]:=
MemoryInUse[]

Plus @@ Table[Function[yy, ReleaseHold[#]][y[[i]]], {i, Length[y]}] &[
    Hold[g][f[#, yy, z] & /@ x]] // Timing

MaxMemoryUsed[]


Out[2]= 1275144
Out[3]= {3.726 Second, \
2822031643462133028498570863037391848660317984511492630514848928370483422063
31\
7366263171941177885366326198093393014236474677191722872982667765803378181410
28\
11184114129504125276977313333635200400}
Out[4]= 1358768
In[5]:= Quit[]

(I finally applied Plus just to check for check for correctness)



(2) same with F (called g here) not held:

Allan Hayes (not held)

In[2]:=
MemoryInUse[]

Plus@@Table[Function[yy,#][y[[i]]],{i,Length[y]}]&[g[f[#,yy,z]&/@x]]//Timing

MaxMemoryUsed[]



Out[2]= 1274896

Out[3]= {0.261 Second, =dito= }

Out[4]= 1350856
 
This is very fast (a little bit surprisingly to me).


(3) The mapping over x (and evaluation of g) certainly should be done
outside of the "inner loop" so to say. So this must be much slower:

In[2]:=
MemoryInUse[]

Plus@@Table[Function[yy,g[f[#,yy,z]&/@x]][y[[i]]],{i,Length[y]}]//Timing

MaxMemoryUsed[]



Out[2]= 1274624

Out[3]= {8.663 Second, =dito=}

Out[4]= 1350856


(4) However I was astonished about this being even slower:

In[2]:=
MemoryInUse[]

Plus@@Table[g[f[#,y[[i]],z]&/@x],{i,Length[y]}]//Timing

MaxMemoryUsed[]



Out[2]= 1274256

Out[3]= {14.792 Second, =dito=}

Out[4]= 1350856

Have you got an idea why?


(5) A natural idea would be to use nested Maps:

In[2]:=
MemoryInUse[]

(Plus @@ Map[Function[{y}, g[Map[Function[{x}, f[x, y, z]], x]]], 
        y]) // Timing

MaxMemoryUsed[]


Out[2]= 1274736
Out[3]= {12.248 Second, =dito= }
Out[4]= 1358512

While this is economic on memory, performance is bad (on grounds stated
above). 


(6) Evaluating the body of the Function definition improves that
considerably:

In[2]:=
MemoryInUse[]

Plus @@ Map[Function[{yy}, Evaluate[g[Map[Function[{x}, f[x, yy, z]], x]]]],

      y] // Timing

MaxMemoryUsed[]

Out[2]= 1274600
Out[3]= {0.241 Second, =dito= }
Out[4]= 1350856


(7) The best result I attained, however was using Thread instead of the
inner Map:

In[2]:=
MemoryInUse[]

Plus @@ (Evaluate[g[Thread[f[x, #, z]]]] &) /@ y // Timing

MaxMemoryUsed[]


Out[2]= 1274040
Out[3]= {0.181 Second, =dito= }
Out[4]= 1350856


yours, Hartmut



  • Prev by Date: Re: problems with ContourPlot
  • Next by Date: Re: Solve-Problem!
  • Previous by thread: Re: Particular structure 2
  • Next by thread: Re: Re: Particular structure 2