Re: "In progress" saving of data collected using Reap/Sow

*To*: mathgroup at smc.vnet.net*Subject*: [mg66164] Re: "In progress" saving of data collected using Reap/Sow*From*: "Ray Koopman" <koopman at sfu.ca>*Date*: Tue, 2 May 2006 02:43:21 -0400 (EDT)*References*: <e2v5pu$n64$1@smc.vnet.net>*Sender*: owner-wri-mathgroup at wolfram.com

giacomo.ciani at gmail.com wrote: > Hello to everyone, > > I'm quite new to Mathematica, so please be patient if I ask something > trivial. > I wrote a little simulation program that need to iterate a calculation > thousand of times, storing the result of each iteration. At first, I > used the "Append" function to store data in a list at every iteration, > but I noticed that this operation become slower and slower as the list > increases in size, so that appending a single data at the end of a > quite big list takes a lot of time. > So I migrated to the reap/sow functions that are written exactly for > this purpose, ad gives much better performances. My code now is > something like: > > results = Reap[Do[a lot of iterations with almost a call to Sow in > each]]]; > > The problem now is that, since the collected data are returned by the > Reap function only whern it terminates, I have no way to access that > data if the calculation is in progress, ie if the Do loop is not > completed. This way, I can't check the status of the calculation nor > provide backup save of data during the calculation itself, that can > last for days... > As you can imagine, using the Append function both this (and others) > tasks where easily accomplishable, since in every moment I had a list > whith all the results obtained so far... > > Any idea on how I can solve the problem? > > Thanks a lot > > Giacomo In[1]:= n = 1*^4 Timing[y = {}; Do[AppendTo[y,i],{i,n}];] Timing[z = {}; Do[z = {z,i},{i,n}];] i = Random[Integer,{1,n}] y[[i]] Last@Nest[First,z,n-i] Out[1]= 10000 Out[2]= {1.86 Second,Null} Out[3]= {0.02 Second,Null} Out[4]= 9353 Out[5]= 9353 Out[6]= 9353