Re: Appending to Lists

• To: mathgroup at smc.vnet.net
• Subject: [mg27062] Re: [mg27045] Appending to Lists
• From: Daniel Lichtblau <danl at wolfram.com>
• Date: Sat, 3 Feb 2001 04:58:56 -0500 (EST)
• References: <200102010800.DAA04717@smc.vnet.net>
• Sender: owner-wri-mathgroup at wolfram.com

```James Jones wrote:
>
> Hi,
>
> I have a function that creates a list (a) from another list (b). The list
> elements are re-grouped in the new list according to a third list (c). A
> Position command is applied to list (c) for an element, then with this
> output the list (a) is created from list (b) at positions given by the
> element position data, list (c). This is repeated for the large number of
> elements in the original lists.
> The Position command is necessary as different elements appear in the list a
> different number of times.
> However, with the large number of elements in the lists (approx 50,000 for a
> simple list), this method is _very_ slow.
> If any one can give me help in speeding this process up I would be very
> grateful.
>
> The data sets would look like this
>
>       b                       c
>
>     0.2                      1
>     0.6                      2
>     1.2                      3
>     -0.2                     1
>     0.5                       2
>     0.3                       1
>     0.7                       2
>    -0.2                      1
>    -0.6                      1
>
> A List would then be created from this data ( the list (a) ) containing
> vectors for 1, 2 and 3. The data in (b) is not important, and the order in
> which elements in (c) drop out is not set.
> In this case the (a) list should look like
>
> a = { { 0.2, -0.2, -0.2, -0.6} , {0.6, 0.5, 0.7} , { 1.2 } }
>
> My current function looks like this
>
> Do[AppendTo[xfinal,
>       Flatten[Part[X, #] & /@
>           Position[Global`PARTICLE, i]]], {i, 1,
>       Max[PARTICLE]}];
>
> where xfinal is an (a) list, i.e. to be created.
>           X is the (b) list , i.e. to be addressed, and
>           PARTICLE is the (c) list. It is referenced by number.
>
> and it is very slow!
>
> Also, after producing this list, the different vector elements need to be
> made the same length, and so 0.0 are added to the ends of all vector
> elements shorter than the longest. My current function for doing this looks
> like
>
> table = Table[0.0, {Length[First[long]]}]; Print["Table Created!"];
>
> Do[If[Length[Part[xfinal, i]] < Length[First[long]],
>       AppendTo[Part[xfinal, i],
>         Drop[table, (Length[Part[xfinal, i]])] ]], {i, 2,
>       Length[xfinal]}];
>
> where list (long) just sorts the list elements according to length.
>
> This function is also very slow, and I was wondering, again, if anyone knew
> a faster way of implementing this. Is the production of a table, once, and
> then dropping bits off and appending the fastest method? Of course this
> needs to be done tens of thousands of times per set of data so any small
> speed increase would be very helpful ;->
>
> Again, any help much appreciated,
>
> James Jones
> Daresbury Laboratory

The notes for Append explain how to work around its speed deficiencies.
Below is some code that will do this stuff in a more functional manner.
One caveat is that it will reorder elements in each bin; I believe you
indicate that that is allowed.

Module[{new, mlen},
new = Split[Sort[Transpose[{bins,data}]], #1[[1]]===#2[[1]]&];
new = Map[#[[2]]&, new, {2}];
mlen = Max[Map[Length, new]];
]

In[100]:= bb = {.2,.6,1.2,-.2,.5,.3,.7,-.2,-.6};

In[101]:= cc = {1,2,3,1,2,1,2,1,1};

Out[102]= {{-0.6, -0.2, -0.2, 0.2, 0.3}, {0.5, 0.6, 0.7, 0, 0},
>    {1.2, 0, 0, 0, 0}}

To get an idea of speed:

n = 10^5;
data = Table[Random[], {n}];
bins = Table[Random[Integer, {1,100}], {n}];

Out[106]= {8.72 Second, Null}

If there are relatively few bins there may be faster ways to do this.

Daniel Lichtblau
Wolfram Research

```

• Prev by Date: Problems rendering Graphics3D with RealTime3D
• Next by Date: variable projection & separable least-squares..
• Previous by thread: Re: Appending to Lists
• Next by thread: Re: Appending to Lists