Re: Estimating slope from noisy data

• To: mathgroup at smc.vnet.net
• Subject: [mg89647] Re: Estimating slope from noisy data
• From: dh <dh at metrohm.ch>
• Date: Mon, 16 Jun 2008 06:40:17 -0400 (EDT)
• References: <g3034r\$mbl\$1@smc.vnet.net>

```
Hi Andreas,

if you want to estimate the slope, it is not very good to sample around

zero. E.g. if you change

to:

Table[rnd[RandomReal[NormalDistribution[1, xrange/3.0]]], {2000}]

you will get a much better estimate.

hope this helps, Daniel

andreas.kohlmajer at gmx.de wrote:

> Hi!

>

> I have difficulties to estimate the correct slope from noisy data.

> This is the code to generate the noisy data:

>

> Needs["LinearRegression`"];

> slope = 1.0;

> sigma = 0.5;

> xrange = 1.0;

>

> SeedRandom[123]; (* initialize random generator *)

> rnd = {#, #*slope + RandomReal[NormalDistribution[0, sigma]]} &;

>

> (* generate 2000 data points *)

> data = Table[

>    rnd[RandomReal[NormalDistribution[0, xrange/3.0]]], {2000}];

>

> subset = Take[data, 8];

> ListPlot[subset, PlotRange -> {{-3, 3}, {-3, 3}},

>  PlotStyle -> PointSize[.025]]

> fit = Regress[subset, x, x, IncludeConstant -> False,

>   RegressionReport -> {SummaryReport, ParameterCITable}]

>

> The correct slope is exactly 1. As the data is quite noisy, the CI of

> the slope is very big. The estimated slope is far to big (1.947). If I

> use more data points, the estimation gets better; I could also use a

> wider x-range, to get a better estimate for the slope. However, I'm

> quite limited in the x-range, so using a wider x-range is no option

> for me.

>

> I could check the RSquared for significance (If[Abs[r*Sqrt[n - 2]/

> Sqrt[1 - r^2]] >=

>   Quantile[StudentTDistribution[n - 2], 1 - 0.05], r, 0] (*

> significance of 95% *)). I this case, it is significant.

>

> Is there any other way to get a good estimate for the slope, without

> using too many data points?

>

>

> (Keywords: fit, regression, slope, noisy, rsquared, limited data)

>

--

Daniel Huber

Metrohm Ltd.

Oberdorfstr. 68

CH-9100 Herisau

Tel. +41 71 353 8585, Fax +41 71 353 8907

E-Mail:<mailto:dh at metrohm.com>

Internet:<http://www.metrohm.com>

```

• Prev by Date: Re: Re: Re: Re: 6.0.3
• Next by Date: Re: How complete a TeX system is Mathematica?