RE: RE: grouping and averaging {x,y} pairs of data
- To: mathgroup at smc.vnet.net
- Subject: [mg37240] RE: [mg37226] RE: grouping and averaging {x,y} pairs of data
- From: "DrBob" <drbob at bigfoot.com>
- Date: Fri, 18 Oct 2002 05:16:59 -0400 (EDT)
- Reply-to: <drbob at bigfoot.com>
- Sender: owner-wri-mathgroup at wolfram.com
I timed Dave's solution versus mine. His is almost three times faster
--- not as a poor a showing for my simple-minded method as I feared.
data2 = Table[{Random[Integer, {1, 100}], Random[Real, {0, 5}]},
{20000}];
nmax = 100;
data = data2;
Timing[dave = Transpose[{Range[nmax], (Plus @@ #1/Length[#1] &) /@
Map[Last, \
Split[Union[Join[data, Table[{i, 0}, {i, 1,
nmax}]]], #1[[1]] == #2[[1]] &], {2}]}];]
ClearAll[total, count]
total[x_] := 0
count[x_] := 0
Timing[{total@#[[1]] += #[[2]], count[#[[1]]]++} & /@ data;
brt = {#, total[#]/count[#]} & /@ Union[data[[All, 1]]];
]
{0.20299999999999985*Second, Null}
{0.5619999999999998*Second, Null}
DrBob
-----Original Message-----
From: David Park [mailto:djmp at earthlink.net]
To: mathgroup at smc.vnet.net
Subject: [mg37240] [mg37226] RE: grouping and averaging {x,y} pairs of data
David,
You will probably get a lot of answers for this. Here is my entry.
data = {{3, 1}, {4, 3}, {3, 2}, {1, 10}, {4, 2}, {1, 6}, {5, 2}, {2, 5},
{7,
1}};
First I will show it step-by-step.
nmax = 10;
Union[Join[data, Table[{i, 0}, {i, 1, nmax}]]]
Split[%, #1[[1]] == #2[[1]] & ]
Map[Last, %, {2}]
(Plus @@ #1/Length[#1] & ) /@ %
Transpose[{Range[nmax], %}]
giving
{{1, 0}, {1, 6}, {1, 10}, {2, 0}, {2, 5}, {3, 0}, {3, 1}, {3, 2}, {4,
0},
{4,
2}, {4, 3}, {5, 0}, {5, 2}, {6, 0}, {7, 0}, {7, 1}, {8, 0}, {9, 0},
{10,
0}}
{{{1, 0}, {1, 6}, {1, 10}}, {{2, 0}, {2, 5}}, {{3, 0}, {3, 1}, {3, 2}},
{{4,
0}, {4, 2}, {4, 3}}, {{5, 0}, {5, 2}}, {{6, 0}}, {{7, 0}, {7, 1}},
{{8,
0}}, {{9, 0}}, {{10, 0}}}
{{0, 6, 10}, {0, 5}, {0, 1, 2}, {0, 2, 3}, {0, 2}, {0}, {0, 1}, {0},
{0},
{0}}
{16/3, 5/2, 1, 5/3, 1, 0, 1/2, 0, 0, 0}
{{1, 16/3}, {2, 5/2}, {3, 1}, {4, 5/3}, {5, 1}, {6, 0},
{7, 1/2}, {8, 0}, {9, 0}, {10, 0}}
This wraps it into one statement.
nmax = 10;
Transpose[{Range[nmax], (Plus @@ #1/Length[#1] & ) /@
Map[Last, Split[Union[Join[data, Table[{i, 0},
{i, 1, nmax}]]], #1[[1]] == #2[[1]] & ], {2}]}]
{{1, 16/3}, {2, 5/2}, {3, 1}, {4, 5/3}, {5, 1}, {6, 0},
{7, 1/2}, {8, 0}, {9, 0}, {10, 0}}
This times a case of 20000 pairs on an 800MHz machine.
data2 = Table[{Random[Integer, {1, 100}], Random[Real, {0, 5}]},
{20000}];
nmax = 100;
data = data2;
Timing[Transpose[{Range[nmax],
(Plus @@ #1/Length[#1] & ) /@ Map[Last,
Split[Union[Join[data, Table[{i, 0},
{i, 1, nmax}]]], #1[[1]] == #2[[1]] & ],
{2}]}]; ]
{0.55 Second, Null}
David Park
djmp at earthlink.net
http://home.earthlink.net/~djmp/
From: David E. Burmaster [mailto:deb at alceon.com]
To: mathgroup at smc.vnet.net
Dear Fellows in MathGroup,
I have a list of 17,000+ {x,y} pairs of data
each x value is a positive integer from 1 to 100+
each y value is a positive real number
As a *short* example, let's consider:
data = {{3,1},{4,3},{3,2},{1,10},{4,2},{1,6},{5,2},{2,5},{7,1}}
I want to group the data by the x value and report the arithmetic
average
of the y values in each group.
For the example, i want to report:
output = {{1,8},{2,5},{3,1.5},{4,2.5},{5,2},{6,0},{7,1}}
In this example, x=6 does not occur so i report the average y[6] = 0.
Can anyone suggest a way to do this efficiently?/
many thanks
dave
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
David E. Burmaster, Ph.D.
Alceon Corporation
POBox 382069 (new Box number effective 1 Sep 2001)
Harvard Square Station
Cambridge, MA 02238-2069 (new ZIP code effective 1 Sep 2001)
Voice 617-864-4300
Web http://www.Alceon.com
Email deb at Alceon.com
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++