RE: AppendTo VERY slow
- To: mathgroup at smc.vnet.net
- Subject: [mg35353] RE: AppendTo VERY slow
- From: "DrBob" <majort at cox-internet.com>
- Date: Tue, 9 Jul 2002 06:49:13 -0400 (EDT)
- Reply-to: <drbob at bigfoot.com>
- Sender: owner-wri-mathgroup at wolfram.com
However, a far better answer is a small subset of the one I give below:
Transpose[{xlist, ylist}]
Done in practically zero time.
Bobby
-----Original Message-----
From: DrBob [mailto:majort at cox-internet.com]
To: mathgroup at smc.vnet.net
Subject: [mg35353] RE: AppendTo VERY slow
That use of Fold doesn't actually USE the capabilities of Fold. The
function f ignores its first argument and you're not using the matrix of
interest as its third argument either, so you're just using Fold as a Do
loop. Here are timings for the AppendTo method, yours, and a true use
of Fold:
n = 10000;
xlist = Table[Random[Integer, {0, 9}], {n}];
ylist = Table[Random[Integer, {0, 9}], {n}];
app := (outlist = {};
Do[elem = {xlist[[i]], ylist[[i]]};
AppendTo[outlist, elem], {i, 1, n}])
Timing[app;]
folder := Partition[Flatten[
Fold[{#1, #2} &, {}, Transpose[{xlist, ylist}]]], 2]
Timing[folder;]
f[x_, k_] := garza = {garza, {xlist[[k]], ylist[[k]]}}
garza = {}; garza = Partition[Flatten[Fold[f, {}, Range[n]]], 2]; //
Timing
outlist == folder == garza
{9.219 Second,Null}
{0.047 Second,Null}
{0.094 Second,Null}
True
Fold is about twice as fast as your Do loop. Here's a larger test:
n = 40000;
xlist = Table[Random[Integer, {0, 9}], {n}];
ylist = Table[Random[Integer, {0, 9}], {n}];
Timing[folder;]
Timing[garza = {}; garza = Partition[Flatten[Fold[f, {}, Range[n]]],
2];]
folder == garza
{0.172 Second,Null}
{0.438 Second,Null}
True
Bobby Treat
-----Original Message-----
From: tgarza01 at prodigy.net.mx [mailto:tgarza01 at prodigy.net.mx]
To: mathgroup at smc.vnet.net
Subject: [mg35353] AppendTo VERY slow
Indeed, AppendTo is an awfully slow function. It takes exponentially
longer
to execute with the list size. I would suggest a simple outlist =
{outlist,
elem} with a Flatten and Partition at the end. For example,
In[1]:=
xlist=Table[Random[Integer,{0,9}],{10000}];
ylist=Table[Random[Integer,{0,9}],{10000}];
I define a function to do things with Append To:
In[2]:=
app[n_]:=(
outlist={};Do[elem={xlist[[i]],ylist[[i]]};
AppendTo[outlist,elem],{i,1,n}
])
In[3]:=
app[10000]//Timing
Out[3]=
{45.37 Second,Null}
It takes 45 seconds to process 10,000 elements. Now, with the approach I
suggest, I define the function f (in order to use Fold, a very powerful
function):
In[4]:=
f[x_,n_]:=outlist={outlist,{xlist[[n]],ylist[[n]]}}
In[5]:=
outlist={};Partition[Flatten[Fold[f,{},Range[10000]]],2];//Timing
Out[5]=
{0.49 Second,Null}
which produces the same result almost 100 times faster.
Tomas Garza
Mexico City
Original Message:
-----------------
From: M.P.Croucher at Sheffield.ac.uk (Mike)
To: mathgroup at smc.vnet.net
Subject: [mg35353] AppendTo VERY slow
I use lists a lot in mathematica and tend to use AppendTo[] a lot in
my programs. Recently I wrote a function that i call over and over
again and found that the results were coming very slowly and i MEAN
slowly. I was doing Fourier Transforms and all kinds of stuff so I
put it down to those at first but I have just put in a load of Print
statements just after each part of the function to see what was taking
so long.
I was amazed to see that the Fourier Transforms were so quick and what
was actually taking the time was a part of my function that collected
the results togther in the form I wanted and outputted the result. It
looks like this
Do[
elem = {xlist[[count]], ylist[[count]]]};
AppendTo[outlist, elem];
, {count, 1, number}
];
It seems that as the list grows it gets slower and slower. Any tips
on a way around this would be greatly appreciated (would speed my life
up no end)
Thank
Mike
--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web.com/ .