MathGroup Archive 2002

[Date Index] [Thread Index] [Author Index]

Search the Archive

RE: AppendTo VERY slow

  • To: mathgroup at smc.vnet.net
  • Subject: [mg35348] RE: AppendTo VERY slow
  • From: "DrBob" <majort at cox-internet.com>
  • Date: Tue, 9 Jul 2002 06:48:09 -0400 (EDT)
  • Reply-to: <drbob at bigfoot.com>
  • Sender: owner-wri-mathgroup at wolfram.com

That use of Fold doesn't actually USE the capabilities of Fold.  The
function f ignores its first argument and you're not using the matrix of
interest as its third argument either, so you're just using Fold as a Do
loop.  Here are timings for the AppendTo method, yours, and a true use
of Fold:

n = 10000;
xlist = Table[Random[Integer, {0, 9}], {n}];
ylist = Table[Random[Integer, {0, 9}], {n}];
app := (outlist = {};
    Do[elem = {xlist[[i]], ylist[[i]]};
      AppendTo[outlist, elem], {i, 1, n}])
Timing[app;]
folder := Partition[Flatten[
    Fold[{#1, #2} &, {}, Transpose[{xlist, ylist}]]], 2]
Timing[folder;]
f[x_, k_] := garza = {garza, {xlist[[k]], ylist[[k]]}}
garza = {}; garza = Partition[Flatten[Fold[f, {}, Range[n]]], 2]; //
Timing
outlist == folder == garza

{9.219 Second,Null}

{0.047 Second,Null}

{0.094 Second,Null}

True

Fold is about twice as fast as your Do loop.  Here's a larger test:

n = 40000;
xlist = Table[Random[Integer, {0, 9}], {n}];
ylist = Table[Random[Integer, {0, 9}], {n}];
Timing[folder;]
Timing[garza = {}; garza = Partition[Flatten[Fold[f, {}, Range[n]]],
2];]
folder == garza

{0.172 Second,Null}

{0.438 Second,Null}

True

Bobby Treat

-----Original Message-----
From: tgarza01 at prodigy.net.mx [mailto:tgarza01 at prodigy.net.mx] 
To: mathgroup at smc.vnet.net
Subject: [mg35348]  AppendTo VERY slow

Indeed, AppendTo is an awfully slow function. It takes exponentially
longer
to execute with the list size. I would suggest a simple outlist =
{outlist,
elem} with a Flatten and Partition at the end. For example,

In[1]:=
xlist=Table[Random[Integer,{0,9}],{10000}];
ylist=Table[Random[Integer,{0,9}],{10000}];

I define a function to do things with Append To:

In[2]:=
app[n_]:=(
outlist={};Do[elem={xlist[[i]],ylist[[i]]};
AppendTo[outlist,elem],{i,1,n}
])

In[3]:=
app[10000]//Timing

Out[3]=
{45.37 Second,Null}

It takes 45 seconds to process 10,000 elements. Now, with the approach I
suggest, I define the function f (in order to use Fold, a very powerful
function):

In[4]:=
f[x_,n_]:=outlist={outlist,{xlist[[n]],ylist[[n]]}}

In[5]:=
outlist={};Partition[Flatten[Fold[f,{},Range[10000]]],2];//Timing

Out[5]=
{0.49 Second,Null}

which produces the same result almost 100 times faster.

Tomas Garza
Mexico City


Original Message:
-----------------
From:  M.P.Croucher at Sheffield.ac.uk (Mike)
To: mathgroup at smc.vnet.net
Subject: [mg35348]  AppendTo VERY slow


I use lists a lot in mathematica and tend to use AppendTo[] a lot in
my programs.  Recently I wrote a function that i call over and over
again and found that the results were coming very slowly and i MEAN
slowly.  I was doing Fourier Transforms and all kinds of stuff so I
put it down to those at first but I have just put in a load of Print
statements just after each part of the function to see what was taking
so long.

I was amazed to see that the Fourier Transforms were so quick and what
was actually taking the time was a part of my function that collected
the results togther in the form I wanted and outputted the result.  It
looks like this

Do[
    	elem = {xlist[[count]], ylist[[count]]]};
    	AppendTo[outlist, elem];
     , {count, 1, number}
    ];

It seems that as the list grows it gets slower and slower.  Any tips
on a way around this would be greatly appreciated (would speed my life
up no end)


Thank

Mike


--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web.com/ .







  • Prev by Date: RE: Strategy for overly long computations?
  • Next by Date: Re: PDE
  • Previous by thread: RE: Re: AppendTo VERY slow
  • Next by thread: RE: AppendTo VERY slow