Services & ResourcesWolfram ForumsMathGroup Archive

[Date Index] [Thread Index] [Author Index]

Re: Partition of multi dimension list

• To: mathgroup at smc.vnet.net
• Subject: [mg52789] Re: [mg52736] Partition of multi dimension list
• From: DrBob <drbob at bigfoot.com>
• Date: Mon, 13 Dec 2004 04:22:51 -0500 (EST)
• References: <200412111021.FAA06477@smc.vnet.net>
• Reply-to: drbob at bigfoot.com
• Sender: owner-wri-mathgroup at wolfram.com

Here's a much faster method:

sample := Random[Integer, {0, #}] & /@ {5, 7, 63, 12, 1, 4, 3}
c = Array[sample &, 50000];
Timing[one = Flatten[CategoryLists[
c, {_}, {_}, Range[64] - 1, {_}, {0, 1}, {_}, {_}][[1, 1]], 4];]
Timing[two = Sort[
       Last@Reap[Scan[Sow[#, List@#[[{3, 5}]]] &, c], _, List]][[All, -1]];]
one == two

{2.422 Second,Null}
{0.172 Second,Null}
True

c = Array[sample &, 500000];
Timing[one = Flatten[CategoryLists[c, {_}, {_}, Range[
       64] - 1, {_}, {0, 1}, {_}, {_}][[1, 1]], 4];]
Timing[two = Sort[Last@Reap[Scan[Sow[#, List@#[[{3,
             5}]]] &, c], _, List]][[All, -1]];]
one == two

{24.187 Second,Null}
{1.656 Second,Null}
True

So it's 14 to 15 times faster.

BUT... if not all combinations of the 3rd and 5th elements appear, this method doesn't return empty sets at the corresponding positions, so it would return less than 128 sets. There are many ways to fix that after the fact, but here's a way to fix it ahead of time. It adds a dummy vector for each combination, then removes the dummies:

dummies = Flatten[
     Table[{dum, dum, i, dum, j, dum, dum}, {i, 0, 63}, {j, 0, 1}], 1];
c = Array[sample &, 50000];
Timing[one = Flatten[CategoryLists[c, {_}, {_}, Range[64] -
     1, {_}, {0, 1}, {_}, {_}][[1, 1]], 4];]
Timing[two = Rest /@ Sort[Last@Reap[
       Scan[Sow[#, List@#[[{3, 5}]]] &, Join[dummies, c]], _, List]][[
             All, -1]];]
one == two

{2.437 Second,Null}
{0.203 Second,Null}
True

Here's a test in which the empty sets actually appear:

c = Array[sample &, 50];
Timing[one = Flatten[CategoryLists[c, {_}, {_},
       Range[64] - 1, {_}, {0, 1}, {_}, {_}][[1, 1]], 4];]
Timing[two = Rest /@ Sort[Last@Reap[Scan[Sow[#, List@#[[{3, 5}]]] &, Join[
             dummies, c]], _, List]][[All, -1]];]
one == two
two

{0.015 Second,Null}

{0. Second,Null}

True

{{},{},{},{},{},{},{{2,6,3,11,0,1,2}},{},{{1,7,4,6,0,3,3}},{},{{4,1,5,7,0,3,
   1}},{},{},{{3,1,6,3,1,1,2}},{{2,6,7,12,0,2,3}},{},{{3,5,8,6,0,2,0}},{{4,3,8,
       7,1,4,0}},{},{{3,7,9,8,1,2,1},{5,0,9,12,1,0,1},{4,6,9,5,
   1,0,3}},{},{{2,2,10,9,1,0,2}},{},{},{{
   4,2,12,4,0,4,0}},{{4,7,12,8,1,3,1}},{{0,7,
   13,6,0,3,1}},{{4,7,13,6,1,4,2}},{{5,
   0,14,5,0,0,1},{5,4,14,11,0,1,0}},{{5,0,
     14,10,1,1,2}},{},{{3,7,15,11,1,3,1}},{{3,5,16,4,0,4,1}},{{4,5,16,5,1,
   3,2}},{{0,5,17,1,0,4,1},{5,5,17,10,0,4,1}},{},{},{{2,3,18,10,1,1,2}},{},{{
       1,3,19,2,1,1,1}},{{5,2,20,1,0,4,0}},{},{{0,0,21,4,0,4,2}},{{3,0,21,11,
       1,1,2}},{},{},{},{},{},{},{},{},{},{},{},{{5,0,27,2,1,0,
       3}},{},{},{},{},{{1,2,30,10,0,3,1},{4,1,30,
     7,0,2,2}},{},{},{},{{1,5,32,5,0,0,3},{4,7,32,2,0,2,
       0}},{},{},{},{},{},{},{},{},{},{},{},{{2,2,38,10,0,2,0}},{{0,6,38,
       10,1,3,3}},{},{},{},{{0,7,40,9,1,0,1}},{},{{5,
   6,41,2,1,2,3}},{{3,1,42,3,0,3,3}},{},{},{},{},{},{},{},{{3,3,46,7,0,
       1,2}},{},{},{},{},{},{},{},{{0,4,50,8,0,1,3}},{{2,
   7,50,4,1,2,3}},{{5,5,51,3,0,4,3},{5,3,51,3,0,2,0}},{},{},{},{},{{0,1,53,2,1,
       1,3}},{},{},{},{},{},{},{{4,3,57,6,0,2,3},{5,4,57,
   10,0,0,0}},{},{{5,4,58,7,0,1,0}},{},{},{},{{3,6,60,7,0,0,1}},{{3,3,
       60,10,1,0,3}},{{0,3,61,6,0,2,2}},{},{},{},{},{}}

Bobby

On Sat, 11 Dec 2004 05:21:51 -0500 (EST), Bruyndonckx P. <pbruynd at vub.ac.be> wrote:

> I have a list of about 5 million items.  Each item is a list of 7 integer numbers (hence a 5000000x7 array).  I want to partition this array on the values of two
> numbers in each item.  The third number (out of seven) goes from 0 to 63 and the fifth number is either 0 or 1.  So I want to end up with 64*2 partitions.  The
> fastest solution I have found up to now to do this on the list 'c' is Flatten[CategoryLists[c, {_}, {_}, Range[64] - 1, {_}, {0,1}, {_}, {_}][[1, 1]], 4].
>
> Since I am not a real expert in Mathematica, can anybody suggest me a possible faster and/or cleaner  way to achieve this.
>
> Thanks,
>
> Peter
>
>
>
>



-- 
DrBob at bigfoot.com
www.eclecticdreams.net

• References:
  • Partition of multi dimension list
    • From: "Bruyndonckx P." <pbruynd@vub.ac.be>