MathGroup Archive 2006

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: "In progress" saving of data collected using Reap/Sow

  • To: mathgroup at smc.vnet.net
  • Subject: [mg66164] Re: "In progress" saving of data collected using Reap/Sow
  • From: "Ray Koopman" <koopman at sfu.ca>
  • Date: Tue, 2 May 2006 02:43:21 -0400 (EDT)
  • References: <e2v5pu$n64$1@smc.vnet.net>
  • Sender: owner-wri-mathgroup at wolfram.com

giacomo.ciani at gmail.com wrote:
> Hello to everyone,
>
> I'm quite new to Mathematica, so please be patient if I ask something
> trivial.
> I wrote a little simulation program that need to iterate a calculation
> thousand of times, storing the result of each iteration. At first, I
> used the "Append" function to store data in a list at every iteration,
> but I noticed that this operation become slower and slower as the list
> increases in size, so that appending a single data at the end of a
> quite big list takes a lot of time.
> So I migrated to the reap/sow functions that are written exactly for
> this purpose, ad gives much better performances. My code now is
> something like:
>
> results = Reap[Do[a lot of iterations with almost a call to Sow in
> each]]];
>
> The problem now is that, since the collected data are returned by the
> Reap function only whern it terminates, I have no way to access that
> data if the calculation is in progress, ie if the Do loop is not
> completed. This way, I can't check the status of the calculation nor
> provide backup save of data during the calculation itself, that can
> last for days...
> As you can imagine, using the Append function both this (and others)
> tasks where easily accomplishable, since in every moment I had a list
> whith all the results obtained so far...
>
> Any idea on how I can solve the problem?
>
> Thanks a lot
>
> Giacomo


In[1]:= n = 1*^4
        Timing[y = {}; Do[AppendTo[y,i],{i,n}];]
        Timing[z = {}; Do[z = {z,i},{i,n}];]
        i = Random[Integer,{1,n}]
        y[[i]]
        Last@Nest[First,z,n-i]

Out[1]= 10000
Out[2]= {1.86 Second,Null}
Out[3]= {0.02 Second,Null}
Out[4]= 9353
Out[5]= 9353
Out[6]= 9353


  • Prev by Date: Do Mathematica applications EVER get updated?
  • Next by Date: Re: Conditions with Statistical Functions
  • Previous by thread: Re: Re: Re: Do Mathematica applications EVER get updated?
  • Next by thread: Re: "In progress" saving of data collected using Reap/Sow