RE: Re: Particular structure 2
- To: mathgroup at smc.vnet.net
- Subject: [mg33657] RE: [mg33627] Re: Particular structure 2
- From: "Wolf, Hartmut" <Hartmut.Wolf at t-systems.com>
- Date: Thu, 4 Apr 2002 19:40:43 -0500 (EST)
- Sender: owner-wri-mathgroup at wolfram.com
Please see at bottom of message > -----Original Message----- > From: Allan Hayes [mailto:hay at haystack.demon.co.uk] To: mathgroup at smc.vnet.net > Sent: Thursday, April 04, 2002 1:09 AM > To: mathgroup at smc.vnet.net > Subject: [mg33657] [mg33627] Re: Particular structure 2 > > > Yas emailed me privately asking for a less memory demanding > soluton than my > earlier on using Outer. > Here is my reply > > How about this (Quit needs to be evaluated each time > separately - or you > could quit the kernel from the kernel menu) > > New code > > Quit > > x= y=Table[1,{400}];f=Plus;F= Times@@#&; > > Table[Function[kk,ReleaseHold[#]][y[[i]]],{i,Length[y]}]& [ > Hold[F][f[#,kk,z]&/@x]];//Timing > > {15.21 Second,Null} > > MaxMemoryUsed[ ] > > 1385656 > > Old code > > Quit > > x= y=Table[1,{400}];f=Plus;F= Times@@#&; > > F/@Map[First,Transpose[Outer[f,x,y,{z}]],{2}];//Timing > > {29.11 Second,Null} > > MaxMemoryUsed[ ] > > 19250032 > Compare to 1385656 > > > > > -- > Allan > > --------------------- > Allan Hayes > Mathematica Training and Consulting > Leicester UK > www.haystack.demon.co.uk > hay at haystack.demon.co.uk > Voice: +44 (0)116 271 4198 > Fax: +44 (0)870 164 0565 > > > "Allan Hayes" <hay at haystack.demon.co.uk> wrote in message > news:a86k5e$jm4$1 at smc.vnet.net... > > Yas, > > > > F/@Map[First,Transpose[Outer[f,{x1,x2,x3},{y1,y2,y3,y4},{z}]],{2}] > > > > > {F[{f[x1,y1,z],f[x2,y1,z],f[x3,y1,z]}],F[{f[x1,y2,z],f[x2,y2,z > ],f[x3,y2,z]}] > > , > > > > > F[{f[x1,y3,z],f[x2,y3,z],f[x3,y3,z]}],F[{f[x1,y4,z],f[x2,y4,z] > ,f[x3,y4,z]}]} > > > > Where F is the function to be used at the second stage. > > > > -- > > Allan > > > > --------------------- > > Allan Hayes > > Mathematica Training and Consulting > > Leicester UK > > www.haystack.demon.co.uk > > hay at haystack.demon.co.uk > > Voice: +44 (0)116 271 4198 > > Fax: +44 (0)870 164 0565 > > > > > > "Yas" <yast at optushome.com.au> wrote in message > > news:a81jgj$778$1 at smc.vnet.net... > > > G'day, > > > How do I go about achieving the structured list below, > > > > > > {f[x1, y1, z], ... , f[xn, y1, z]} > > > > > > followed by some operation on the above list (e.g Fold), store the > > > result and then, do the same again for other y values, as in, > > > > > > {f[x1, ym, z], ... , f[xn, ym, z]} > > > > > > until the last value of ym. > > > > > > In summary, the n values of x get slotted in first for > one value of y > > > then the resulting list is evaluated, the answer stored, > values of x get > > > slotted in again, y is incremented and so forth until all > (m) values of > > > y have been done. > > > > > > Thanks > > > Yas > > > > > > > > > > > > > > > Dear Allan, I was let to ponder for a while why you held F in the Map over x. Without knowing more details of the problem, a "best solution" can hardly be proposed. Of course there might be cases where the evaluation of F might go astray unless the values y[[i]] are inserted. But normally, in the interest of memory consumption and performance (partial) evaluation should be done as early as possible. Here are some of my observations: (1) reproducing your case above In[1]:= x = y = Table[1, {400}]; f = Plus; g = Times @@ # &; z = 1; In[2]:= MemoryInUse[] Plus @@ Table[Function[yy, ReleaseHold[#]][y[[i]]], {i, Length[y]}] &[ Hold[g][f[#, yy, z] & /@ x]] // Timing MaxMemoryUsed[] Out[2]= 1275144 Out[3]= {3.726 Second, \ 2822031643462133028498570863037391848660317984511492630514848928370483422063 31\ 7366263171941177885366326198093393014236474677191722872982667765803378181410 28\ 11184114129504125276977313333635200400} Out[4]= 1358768 In[5]:= Quit[] (I finally applied Plus just to check for check for correctness) (2) same with F (called g here) not held: Allan Hayes (not held) In[2]:= MemoryInUse[] Plus@@Table[Function[yy,#][y[[i]]],{i,Length[y]}]&[g[f[#,yy,z]&/@x]]//Timing MaxMemoryUsed[] Out[2]= 1274896 Out[3]= {0.261 Second, =dito= } Out[4]= 1350856 This is very fast (a little bit surprisingly to me). (3) The mapping over x (and evaluation of g) certainly should be done outside of the "inner loop" so to say. So this must be much slower: In[2]:= MemoryInUse[] Plus@@Table[Function[yy,g[f[#,yy,z]&/@x]][y[[i]]],{i,Length[y]}]//Timing MaxMemoryUsed[] Out[2]= 1274624 Out[3]= {8.663 Second, =dito=} Out[4]= 1350856 (4) However I was astonished about this being even slower: In[2]:= MemoryInUse[] Plus@@Table[g[f[#,y[[i]],z]&/@x],{i,Length[y]}]//Timing MaxMemoryUsed[] Out[2]= 1274256 Out[3]= {14.792 Second, =dito=} Out[4]= 1350856 Have you got an idea why? (5) A natural idea would be to use nested Maps: In[2]:= MemoryInUse[] (Plus @@ Map[Function[{y}, g[Map[Function[{x}, f[x, y, z]], x]]], y]) // Timing MaxMemoryUsed[] Out[2]= 1274736 Out[3]= {12.248 Second, =dito= } Out[4]= 1358512 While this is economic on memory, performance is bad (on grounds stated above). (6) Evaluating the body of the Function definition improves that considerably: In[2]:= MemoryInUse[] Plus @@ Map[Function[{yy}, Evaluate[g[Map[Function[{x}, f[x, yy, z]], x]]]], y] // Timing MaxMemoryUsed[] Out[2]= 1274600 Out[3]= {0.241 Second, =dito= } Out[4]= 1350856 (7) The best result I attained, however was using Thread instead of the inner Map: In[2]:= MemoryInUse[] Plus @@ (Evaluate[g[Thread[f[x, #, z]]]] &) /@ y // Timing MaxMemoryUsed[] Out[2]= 1274040 Out[3]= {0.181 Second, =dito= } Out[4]= 1350856 yours, Hartmut