MathGroup Archive 2008

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Estimating slope from noisy data

  • To: mathgroup at smc.vnet.net
  • Subject: [mg89647] Re: Estimating slope from noisy data
  • From: dh <dh at metrohm.ch>
  • Date: Mon, 16 Jun 2008 06:40:17 -0400 (EDT)
  • References: <g3034r$mbl$1@smc.vnet.net>


Hi Andreas,

if you want to estimate the slope, it is not very good to sample around 

zero. E.g. if you change

to:

Table[rnd[RandomReal[NormalDistribution[1, xrange/3.0]]], {2000}]

you will get a much better estimate.

hope this helps, Daniel



andreas.kohlmajer at gmx.de wrote:

> Hi!

> 

> I have difficulties to estimate the correct slope from noisy data.

> This is the code to generate the noisy data:

> 

> Needs["LinearRegression`"];

> slope = 1.0;

> sigma = 0.5;

> xrange = 1.0;

> 

> SeedRandom[123]; (* initialize random generator *)

> rnd = {#, #*slope + RandomReal[NormalDistribution[0, sigma]]} &;

> 

> (* generate 2000 data points *)

> data = Table[

>    rnd[RandomReal[NormalDistribution[0, xrange/3.0]]], {2000}];

> 

> subset = Take[data, 8];

> ListPlot[subset, PlotRange -> {{-3, 3}, {-3, 3}},

>  PlotStyle -> PointSize[.025]]

> fit = Regress[subset, x, x, IncludeConstant -> False,

>   RegressionReport -> {SummaryReport, ParameterCITable}]

> 

> The correct slope is exactly 1. As the data is quite noisy, the CI of

> the slope is very big. The estimated slope is far to big (1.947). If I

> use more data points, the estimation gets better; I could also use a

> wider x-range, to get a better estimate for the slope. However, I'm

> quite limited in the x-range, so using a wider x-range is no option

> for me.

> 

> I could check the RSquared for significance (If[Abs[r*Sqrt[n - 2]/

> Sqrt[1 - r^2]] >=

>   Quantile[StudentTDistribution[n - 2], 1 - 0.05], r, 0] (*

> significance of 95% *)). I this case, it is significant.

> 

> Is there any other way to get a good estimate for the slope, without

> using too many data points?

> 

> 

> (Keywords: fit, regression, slope, noisy, rsquared, limited data)

> 





-- 



Daniel Huber

Metrohm Ltd.

Oberdorfstr. 68

CH-9100 Herisau

Tel. +41 71 353 8585, Fax +41 71 353 8907

E-Mail:<mailto:dh at metrohm.com>

Internet:<http://www.metrohm.com>




  • Prev by Date: Re: Re: Re: Re: 6.0.3
  • Next by Date: Re: How complete a TeX system is Mathematica?
  • Previous by thread: Re: Problem downloading from UK Wolfram site
  • Next by thread: How to create a style sheet