Re: Problems with DistributionFitTest
- To: mathgroup at smc.vnet.net
- Subject: [mg122591] Re: Problems with DistributionFitTest
- From: Andy Ross <andyr at wolfram.com>
- Date: Wed, 2 Nov 2011 06:23:12 -0500 (EST)
- Delivered-to: l-mathgroup@mail-archive0.wolfram.com
- References: <201111010502.AAA14754@smc.vnet.net>
This is exactly what you might expect. The p-value from a hypothesis test is itself a random variable. Under the null hypothesis the p-value should follow a UniformDistribution[{0,1}]. In your case, the null hypothesis is that the data have been drawn from a normal distribution. What that p-value is really saying is that about 3% of the time you can expect to get a test statistic like the one you obtained or one even more extreme. Andy Ross Wolfram Research On 11/1/2011 12:02 AM, fd wrote: > Dear Group > > I'm not a specialist in statistics, but I spoke to one who found this > behaviour dubious. > > Before using DistributionFitTest I was doing some tests with the > normal distribution, like this > > data = RandomVariate[NormalDistribution[], 10000]; > > DistributionFitTest[data] > > 0.0312946 > > According to the documentation "A small p-value suggests that it is > unlikely that the data came from dist", and that the test assumes the > data is normally distributed > > I found this result for the p-value to be really low, if I re-run the > code I often get what I would expect (a number greater than 0.5) but > it is not at all rare to obtain p values smaller than 0.05 and even > smaller. Through multiple re-runs I notice it fluctuates by orders of > magnitude. > > The statistician I consulted with found this weird since the data was > drawn from a a normal distribution and the sample size is big, > especially because the Pearson X2 test also fluctuates like this: > > H=DistributionFitTest[data, Automatic, "HypothesisTestData"]; > > H["TestDataTable", All] > > Is this a real issue? > > Any thougths > > Best regards > Felipe > > > >