MathGroup Archive 2011

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Problems with DistributionFitTest

  • To: mathgroup at
  • Subject: [mg122591] Re: Problems with DistributionFitTest
  • From: Andy Ross <andyr at>
  • Date: Wed, 2 Nov 2011 06:23:12 -0500 (EST)
  • Delivered-to:
  • References: <>

This is exactly what you might expect.  The p-value from a hypothesis 
test is itself a random variable. Under the null hypothesis the p-value 
should follow a UniformDistribution[{0,1}].

In your case, the null hypothesis is that the data have been drawn from 
a normal distribution. What that p-value is really saying is that about 
3% of the time you can expect to get a test statistic like the one you 
obtained or one even more extreme.

Andy Ross
Wolfram Research

On 11/1/2011 12:02 AM, fd wrote:
> Dear Group
> I'm not a specialist in statistics, but I spoke to one who found this
> behaviour dubious.
> Before using DistributionFitTest I was doing some tests with the
> normal distribution, like this
> data = RandomVariate[NormalDistribution[], 10000];
> DistributionFitTest[data]
> 0.0312946
> According to the documentation "A small p-value suggests that it is
> unlikely that the data came from dist", and that the test assumes the
> data is normally distributed
> I found this result for the p-value to be really low, if I re-run the
> code I often get what I would expect (a number greater than 0.5) but
> it is not at all rare to obtain p values smaller than 0.05 and even
> smaller. Through multiple re-runs I notice it fluctuates by orders of
> magnitude.
> The statistician I consulted with found this weird since the data was
> drawn from a a normal distribution and the sample size is big,
> especially because the Pearson X2 test also fluctuates like this:
> H=DistributionFitTest[data, Automatic, "HypothesisTestData"];
> H["TestDataTable", All]
> Is this a real issue?
> Any thougths
> Best regards
> Felipe

  • Prev by Date: Re: Coin Toss Sim
  • Next by Date: Re: Shifted & scaled Heaviside and FT
  • Previous by thread: Re: Problems with DistributionFitTest
  • Next by thread: Re: Problems with DistributionFitTest