Re: "In progress" saving of data collected using Reap/Sow
- To: mathgroup at smc.vnet.net
- Subject: [mg66179] Re: "In progress" saving of data collected using Reap/Sow
- From: "Ray Koopman" <koopman at sfu.ca>
- Date: Wed, 3 May 2006 02:44:52 -0400 (EDT)
- References: <e2v5pu$n64$1@smc.vnet.net><e36vft$n7c$1@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
Expanding on my previous post, here are three ways to unravel z into
a simple list of all the past results. The first way is fastest, but
doesn't work if each result itself is a list.
In[1]:= n = 8
y = {}; Do[AppendTo[y,result[i]],{i,n}]; y
z = {}; Do[z = {z,result[i]},{i,n}]; z
Flatten[z] === y
Nest[FlattenAt[#,1]&,z,n] === y
Table[Last@Nest[First,z,n-i],{i,n}] === y
Out[1]= 8
Out[2]= {result[1],result[2],result[3],result[4],
result[5],result[6],result[7],result[8]}
Out[3]= {{{{{{{{{},result[1]},result[2]},result[3]},result[4]},
result[5]},result[6]},result[7]},result[8]}
Out[4]= True
Out[5]= True
Out[6]= True
In[7]:= n = 8
y = {}; Do[AppendTo[y,{i,-i}],{i,n}]; y
z = {}; Do[z = {z,{i,-i}},{i,n}]; z
Flatten[z] === y
Nest[FlattenAt[#,1]&,z,n] === y
Table[Last@Nest[First,z,n-i],{i,n}] === y
Out[7]= 8
Out[8]= {{1,-1},{2,-2},{3,-3},{4,-4},
{5,-5},{6,-6},{7,-7},{8,-8}}
Out[9]= {{{{{{{{{},{1,-1}},{2,-2}},{3,-3}},{4,-4}},
{5,-5}},{6,-6}},{7,-7}},{8,-8}}
Out[10]= False
Out[11]= True
Out[12]= True
Ray Koopman wrote:
> giacomo.ciani at gmail.com wrote:
> > Hello to everyone,
> >
> > I'm quite new to Mathematica, so please be patient if I ask something
> > trivial.
> > I wrote a little simulation program that need to iterate a calculation
> > thousand of times, storing the result of each iteration. At first, I
> > used the "Append" function to store data in a list at every iteration,
> > but I noticed that this operation become slower and slower as the list
> > increases in size, so that appending a single data at the end of a
> > quite big list takes a lot of time.
> > So I migrated to the reap/sow functions that are written exactly for
> > this purpose, ad gives much better performances. My code now is
> > something like:
> >
> > results = Reap[Do[a lot of iterations with almost a call to Sow in
> > each]]];
> >
> > The problem now is that, since the collected data are returned by the
> > Reap function only whern it terminates, I have no way to access that
> > data if the calculation is in progress, ie if the Do loop is not
> > completed. This way, I can't check the status of the calculation nor
> > provide backup save of data during the calculation itself, that can
> > last for days...
> > As you can imagine, using the Append function both this (and others)
> > tasks where easily accomplishable, since in every moment I had a list
> > whith all the results obtained so far...
> >
> > Any idea on how I can solve the problem?
> >
> > Thanks a lot
> >
> > Giacomo
>
>
> In[1]:= n = 1*^4
> Timing[y = {}; Do[AppendTo[y,i],{i,n}];]
> Timing[z = {}; Do[z = {z,i},{i,n}];]
> i = Random[Integer,{1,n}]
> y[[i]]
> Last@Nest[First,z,n-i]
>
> Out[1]= 10000
> Out[2]= {1.86 Second,Null}
> Out[3]= {0.02 Second,Null}
> Out[4]= 9353
> Out[5]= 9353
> Out[6]= 9353