RE: RE: grouping and averaging {x,y} pairs of data

*To*: mathgroup at smc.vnet.net*Subject*: [mg37240] RE: [mg37226] RE: grouping and averaging {x,y} pairs of data*From*: "DrBob" <drbob at bigfoot.com>*Date*: Fri, 18 Oct 2002 05:16:59 -0400 (EDT)*Reply-to*: <drbob at bigfoot.com>*Sender*: owner-wri-mathgroup at wolfram.com

I timed Dave's solution versus mine. His is almost three times faster --- not as a poor a showing for my simple-minded method as I feared. data2 = Table[{Random[Integer, {1, 100}], Random[Real, {0, 5}]}, {20000}]; nmax = 100; data = data2; Timing[dave = Transpose[{Range[nmax], (Plus @@ #1/Length[#1] &) /@ Map[Last, \ Split[Union[Join[data, Table[{i, 0}, {i, 1, nmax}]]], #1[[1]] == #2[[1]] &], {2}]}];] ClearAll[total, count] total[x_] := 0 count[x_] := 0 Timing[{total@#[[1]] += #[[2]], count[#[[1]]]++} & /@ data; brt = {#, total[#]/count[#]} & /@ Union[data[[All, 1]]]; ] {0.20299999999999985*Second, Null} {0.5619999999999998*Second, Null} DrBob -----Original Message----- From: David Park [mailto:djmp at earthlink.net] To: mathgroup at smc.vnet.net Subject: [mg37240] [mg37226] RE: grouping and averaging {x,y} pairs of data David, You will probably get a lot of answers for this. Here is my entry. data = {{3, 1}, {4, 3}, {3, 2}, {1, 10}, {4, 2}, {1, 6}, {5, 2}, {2, 5}, {7, 1}}; First I will show it step-by-step. nmax = 10; Union[Join[data, Table[{i, 0}, {i, 1, nmax}]]] Split[%, #1[[1]] == #2[[1]] & ] Map[Last, %, {2}] (Plus @@ #1/Length[#1] & ) /@ % Transpose[{Range[nmax], %}] giving {{1, 0}, {1, 6}, {1, 10}, {2, 0}, {2, 5}, {3, 0}, {3, 1}, {3, 2}, {4, 0}, {4, 2}, {4, 3}, {5, 0}, {5, 2}, {6, 0}, {7, 0}, {7, 1}, {8, 0}, {9, 0}, {10, 0}} {{{1, 0}, {1, 6}, {1, 10}}, {{2, 0}, {2, 5}}, {{3, 0}, {3, 1}, {3, 2}}, {{4, 0}, {4, 2}, {4, 3}}, {{5, 0}, {5, 2}}, {{6, 0}}, {{7, 0}, {7, 1}}, {{8, 0}}, {{9, 0}}, {{10, 0}}} {{0, 6, 10}, {0, 5}, {0, 1, 2}, {0, 2, 3}, {0, 2}, {0}, {0, 1}, {0}, {0}, {0}} {16/3, 5/2, 1, 5/3, 1, 0, 1/2, 0, 0, 0} {{1, 16/3}, {2, 5/2}, {3, 1}, {4, 5/3}, {5, 1}, {6, 0}, {7, 1/2}, {8, 0}, {9, 0}, {10, 0}} This wraps it into one statement. nmax = 10; Transpose[{Range[nmax], (Plus @@ #1/Length[#1] & ) /@ Map[Last, Split[Union[Join[data, Table[{i, 0}, {i, 1, nmax}]]], #1[[1]] == #2[[1]] & ], {2}]}] {{1, 16/3}, {2, 5/2}, {3, 1}, {4, 5/3}, {5, 1}, {6, 0}, {7, 1/2}, {8, 0}, {9, 0}, {10, 0}} This times a case of 20000 pairs on an 800MHz machine. data2 = Table[{Random[Integer, {1, 100}], Random[Real, {0, 5}]}, {20000}]; nmax = 100; data = data2; Timing[Transpose[{Range[nmax], (Plus @@ #1/Length[#1] & ) /@ Map[Last, Split[Union[Join[data, Table[{i, 0}, {i, 1, nmax}]]], #1[[1]] == #2[[1]] & ], {2}]}]; ] {0.55 Second, Null} David Park djmp at earthlink.net http://home.earthlink.net/~djmp/ From: David E. Burmaster [mailto:deb at alceon.com] To: mathgroup at smc.vnet.net Dear Fellows in MathGroup, I have a list of 17,000+ {x,y} pairs of data each x value is a positive integer from 1 to 100+ each y value is a positive real number As a *short* example, let's consider: data = {{3,1},{4,3},{3,2},{1,10},{4,2},{1,6},{5,2},{2,5},{7,1}} I want to group the data by the x value and report the arithmetic average of the y values in each group. For the example, i want to report: output = {{1,8},{2,5},{3,1.5},{4,2.5},{5,2},{6,0},{7,1}} In this example, x=6 does not occur so i report the average y[6] = 0. Can anyone suggest a way to do this efficiently?/ many thanks dave +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ David E. Burmaster, Ph.D. Alceon Corporation POBox 382069 (new Box number effective 1 Sep 2001) Harvard Square Station Cambridge, MA 02238-2069 (new ZIP code effective 1 Sep 2001) Voice 617-864-4300 Web http://www.Alceon.com Email deb at Alceon.com +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++