Re: Regress, Eliminate Outliers, Regress
- To: mathgroup at smc.vnet.net
- Subject: [mg29279] Re: [mg29270] Regress, Eliminate Outliers, Regress
- From: BobHanlon at aol.com
- Date: Mon, 11 Jun 2001 04:38:23 -0400 (EDT)
- Sender: owner-wri-mathgroup at wolfram.com
In a message dated 2001/6/9 3:22:35 AM, Moranresearch at aol.com writes:
>I would like to do a simple linear regression on a data set, eliminate
>data
>points that are > 2 standard deviations away from the regression line,
>then
>repeat the regression on the reduced data set. Thank you. John
>
xmax = 10;
data = Table[{x=Random[Real, {0, xmax} ], 3*x+10+3(2Random[]-1)}, {15}];
Clear[x];
f1[x_] := Evaluate[Fit[data, {1, x}, x]];
f1[x]
3.0631275218404572*x + 10.073785162728084
sd = Sqrt[Plus@@((f1[#[[1]]]-#[[2]])^2& /@ data)/Length[data]];
threshold = 1.25 sd; (* your threshold is 2sd *)
filteredData = Select[data, Abs[f1[#[[1]]]-#[[2]]] <= threshold&];
f2[x_] := Evaluate[Fit[filteredData, {1, x}, x]];
f2[x]
3.0696602316273487*x + 9.752061399750337
Plot[{f2[x], f1[x], f1[x]+threshold, f1[x]-threshold}, {x, 0, xmax},
PlotStyle ->
{{AbsoluteThickness[2], AbsoluteDashing[{2,8, 5, 8}],
RGBColor[1, 0, 0]},
{AbsoluteThickness[2], AbsoluteDashing[{5,8, 2, 8}],
RGBColor[0, 0, 1]},
{AbsoluteDashing[{2,8, 2, 8}]},
{AbsoluteDashing[{2,8, 2, 8}]}},
Prolog ->
{AbsolutePointSize[5], RGBColor[1, 0, 1],
Point /@ Complement[data, filteredData],
AbsolutePointSize[3], RGBColor[0, 0, 0], Point /@ filteredData},
ImageSize -> 500];
Bob Hanlon
Chantilly, VA USA