Re: Appending to Lists
- To: mathgroup at smc.vnet.net
- Subject: [mg27062] Re: [mg27045] Appending to Lists
- From: Daniel Lichtblau <danl at wolfram.com>
- Date: Sat, 3 Feb 2001 04:58:56 -0500 (EST)
- References: <200102010800.DAA04717@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
James Jones wrote:
>
> Hi,
>
> I have a function that creates a list (a) from another list (b). The list
> elements are re-grouped in the new list according to a third list (c). A
> Position command is applied to list (c) for an element, then with this
> output the list (a) is created from list (b) at positions given by the
> element position data, list (c). This is repeated for the large number of
> elements in the original lists.
> The Position command is necessary as different elements appear in the list a
> different number of times.
> However, with the large number of elements in the lists (approx 50,000 for a
> simple list), this method is _very_ slow.
> If any one can give me help in speeding this process up I would be very
> grateful.
>
> The data sets would look like this
>
> b c
>
> 0.2 1
> 0.6 2
> 1.2 3
> -0.2 1
> 0.5 2
> 0.3 1
> 0.7 2
> -0.2 1
> -0.6 1
>
> A List would then be created from this data ( the list (a) ) containing
> vectors for 1, 2 and 3. The data in (b) is not important, and the order in
> which elements in (c) drop out is not set.
> In this case the (a) list should look like
>
> a = { { 0.2, -0.2, -0.2, -0.6} , {0.6, 0.5, 0.7} , { 1.2 } }
>
> My current function looks like this
>
> Do[AppendTo[xfinal,
> Flatten[Part[X, #] & /@
> Position[Global`PARTICLE, i]]], {i, 1,
> Max[PARTICLE]}];
>
> where xfinal is an (a) list, i.e. to be created.
> X is the (b) list , i.e. to be addressed, and
> PARTICLE is the (c) list. It is referenced by number.
>
> and it is very slow!
>
> Also, after producing this list, the different vector elements need to be
> made the same length, and so 0.0 are added to the ends of all vector
> elements shorter than the longest. My current function for doing this looks
> like
>
> table = Table[0.0, {Length[First[long]]}]; Print["Table Created!"];
>
> Do[If[Length[Part[xfinal, i]] < Length[First[long]],
> AppendTo[Part[xfinal, i],
> Drop[table, (Length[Part[xfinal, i]])] ]], {i, 2,
> Length[xfinal]}];
>
> where list (long) just sorts the list elements according to length.
>
> This function is also very slow, and I was wondering, again, if anyone knew
> a faster way of implementing this. Is the production of a table, once, and
> then dropping bits off and appending the fastest method? Of course this
> needs to be done tens of thousands of times per set of data so any small
> speed increase would be very helpful ;->
>
> Again, any help much appreciated,
>
> James Jones
> Daresbury Laboratory
The notes for Append explain how to work around its speed deficiencies.
Below is some code that will do this stuff in a more functional manner.
One caveat is that it will reorder elements in each bin; I believe you
indicate that that is allowed.
binAndPad[data_List, bins_List] /; Length[data]==Length[bins] :=
Module[{new, mlen},
new = Split[Sort[Transpose[{bins,data}]], #1[[1]]===#2[[1]]&];
new = Map[#[[2]]&, new, {2}];
mlen = Max[Map[Length, new]];
Map[PadRight[#, mlen]&, new]
]
In[100]:= bb = {.2,.6,1.2,-.2,.5,.3,.7,-.2,-.6};
In[101]:= cc = {1,2,3,1,2,1,2,1,1};
In[102]:= binAndPad[bb,cc]
Out[102]= {{-0.6, -0.2, -0.2, 0.2, 0.3}, {0.5, 0.6, 0.7, 0, 0},
> {1.2, 0, 0, 0, 0}}
To get an idea of speed:
n = 10^5;
data = Table[Random[], {n}];
bins = Table[Random[Integer, {1,100}], {n}];
In[106]:= Timing[result = binAndPad[data,bins];]
Out[106]= {8.72 Second, Null}
If there are relatively few bins there may be faster ways to do this.
Daniel Lichtblau
Wolfram Research
- References:
- Appending to Lists
- From: "James Jones" <j.k.jones@dl.ac.uk>
- Appending to Lists