Mathematica 9 is now available
Services & Resources / Wolfram Forums
-----
 /
MathGroup Archive
2001
*January
*February
*March
*April
*May
*June
*July
*August
*September
*October
*November
*December
*Archive Index
*Ask about this page
*Print this page
*Give us feedback
*Sign up for the Wolfram Insider

MathGroup Archive 2001

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Appending to Lists

  • To: mathgroup at smc.vnet.net
  • Subject: [mg27062] Re: [mg27045] Appending to Lists
  • From: Daniel Lichtblau <danl at wolfram.com>
  • Date: Sat, 3 Feb 2001 04:58:56 -0500 (EST)
  • References: <200102010800.DAA04717@smc.vnet.net>
  • Sender: owner-wri-mathgroup at wolfram.com

James Jones wrote:
> 
> Hi,
> 
> I have a function that creates a list (a) from another list (b). The list
> elements are re-grouped in the new list according to a third list (c). A
> Position command is applied to list (c) for an element, then with this
> output the list (a) is created from list (b) at positions given by the
> element position data, list (c). This is repeated for the large number of
> elements in the original lists.
> The Position command is necessary as different elements appear in the list a
> different number of times.
> However, with the large number of elements in the lists (approx 50,000 for a
> simple list), this method is _very_ slow.
> If any one can give me help in speeding this process up I would be very
> grateful.
> 
> The data sets would look like this
> 
>       b                       c
> 
>     0.2                      1
>     0.6                      2
>     1.2                      3
>     -0.2                     1
>     0.5                       2
>     0.3                       1
>     0.7                       2
>    -0.2                      1
>    -0.6                      1
> 
> A List would then be created from this data ( the list (a) ) containing
> vectors for 1, 2 and 3. The data in (b) is not important, and the order in
> which elements in (c) drop out is not set.
> In this case the (a) list should look like
> 
> a = { { 0.2, -0.2, -0.2, -0.6} , {0.6, 0.5, 0.7} , { 1.2 } }
> 
> My current function looks like this
> 
> Do[AppendTo[xfinal,
>       Flatten[Part[X, #] & /@
>           Position[Global`PARTICLE, i]]], {i, 1,
>       Max[PARTICLE]}];
> 
> where xfinal is an (a) list, i.e. to be created.
>           X is the (b) list , i.e. to be addressed, and
>           PARTICLE is the (c) list. It is referenced by number.
> 
> and it is very slow!
> 
> Also, after producing this list, the different vector elements need to be
> made the same length, and so 0.0 are added to the ends of all vector
> elements shorter than the longest. My current function for doing this looks
> like
> 
> table = Table[0.0, {Length[First[long]]}]; Print["Table Created!"];
> 
> Do[If[Length[Part[xfinal, i]] < Length[First[long]],
>       AppendTo[Part[xfinal, i],
>         Drop[table, (Length[Part[xfinal, i]])] ]], {i, 2,
>       Length[xfinal]}];
> 
> where list (long) just sorts the list elements according to length.
> 
> This function is also very slow, and I was wondering, again, if anyone knew
> a faster way of implementing this. Is the production of a table, once, and
> then dropping bits off and appending the fastest method? Of course this
> needs to be done tens of thousands of times per set of data so any small
> speed increase would be very helpful ;->
> 
> Again, any help much appreciated,
> 
> James Jones
> Daresbury Laboratory


The notes for Append explain how to work around its speed deficiencies.
Below is some code that will do this stuff in a more functional manner.
One caveat is that it will reorder elements in each bin; I believe you
indicate that that is allowed.

binAndPad[data_List, bins_List] /; Length[data]==Length[bins] :=
	Module[{new, mlen},
	new = Split[Sort[Transpose[{bins,data}]], #1[[1]]===#2[[1]]&];
	new = Map[#[[2]]&, new, {2}];
	mlen = Max[Map[Length, new]];
	Map[PadRight[#, mlen]&, new]
	]

In[100]:= bb = {.2,.6,1.2,-.2,.5,.3,.7,-.2,-.6};

In[101]:= cc = {1,2,3,1,2,1,2,1,1};

In[102]:= binAndPad[bb,cc]
Out[102]= {{-0.6, -0.2, -0.2, 0.2, 0.3}, {0.5, 0.6, 0.7, 0, 0}, 
>    {1.2, 0, 0, 0, 0}}

To get an idea of speed:

n = 10^5;
data = Table[Random[], {n}];
bins = Table[Random[Integer, {1,100}], {n}];

In[106]:= Timing[result = binAndPad[data,bins];]
Out[106]= {8.72 Second, Null}

If there are relatively few bins there may be faster ways to do this.


Daniel Lichtblau
Wolfram Research


  • Prev by Date: Problems rendering Graphics3D with RealTime3D
  • Next by Date: variable projection & separable least-squares..
  • Previous by thread: Re: Appending to Lists
  • Next by thread: Re: Appending to Lists