[Date Index] [Thread Index] [Author Index]

Precision & Accuracy: SUMMARY

• To: mathgroup at christensen.cybernetics.net
• Subject: [mg743] Precision & Accuracy: SUMMARY
• From: Paul E Howland <PEHOWLAND at taz.dra.hmg.gb>
• Date: Wed, 12 Apr 1995 15:43:49 +0200 (IST)

Mathgroup,

I posted a question yesterday regarding the difference between Accuracy[] 
and Precision[] in Mathematica.  I had two different but interesting 
replies, which I summarise below.  To solve my problem, though, I 
eventually delved into the package NonlinearFit.m to see how the Pro's did 
it: I'll also mention my findings about this here.

The first answer provided an analogy with playing darts.  Accuracy 
implies that the average position of the darts is near the bull's eye, 
although their variance can be quite high, whereas Precision implies that 
all the darts hit the same place, but this may be nowhere near the bull's 
eye.  That is, Accuracy=Unbiased answer, Precision=Low Variance answer.  

The second answer provided a useful example of where the Accuracy and 
Precision of a number vary:
 Accuracy[1.00000000000000000001 * 10^-5]        = 25
 Precision[1.00000000000000000001 * 10^-5]       = 20

I was still left wondering how this could be applied when optimising a 
function.  What is the difference between the accuracy of my optimisation 
and its precision?   I had a delve into the code of NonlinearFit.m, and 
found the answer.  When trying to minimise a cost function (NonlinearFit 
uses the Chi squared function), the function satisfies the accuracy 
constraint if:
	0 <= chisq-newchi < (10^-AccuracyGoal)
but satisfies the precision constraint if:
	0 <= chisq-newchi < newchi*(10^-PrecisionGoal)
where chisq is the value of the cost function at the last iteration, and 
newchi is the value of the cost function at the current iteration.

Thus AccuracyGoal specifies that the algorithm be declared converged when
the _absolute_ difference in the cost function is less than a certain
threshold, whereas PrecisionGoal specifies that the algorithm be declared
converged when the _proportional_ change in the cost function is less 
than a specified threshold.

*** TWO WARNINGS ABOUT NonlinearFit.m ***
=========================================

1.	Whilst looking at the code for NonlinearFit, I noticed that the
default values for AccuracyGoal and PrecisionGoal are:
	AccuracyGoal -> 1
	PrecisionGoal -> 3
rather than
	AccuracyGoal -> $MachinePrecision-10
	PrecisionGoal -> $MachinePrecision-10
as stated in the "Guide to Standard Mathematica Packages V2.2".  Thus 
results may be considerably less precise than you expect!

2.	Be wary of the message "NonlinearFit::lmpnocon" which warns that the
values of the parameters do not appear to have converged.  The algorithm
produces this warning if the difference between the previous value of a
parameter and its new value is greater than 1 (for any of the parameters),
at the end of the calculation.  It will thus be _highly_ dependent on the
units of the parameters you are fitting!  Be warned! 

Paul E Howland                         

Long Range Ground Radar Systems Section               tel. +44 (0)1684 895767
CSS2 Division, Room BY209                             fax. +44 (0)1684 896315
Defence Research Agency                           email: PEHOWLAND at DRA.HMG.GB 
Malvern, Worcs, WR14 3PS, UK.                      
-----------------------------------------------------------------------------