Precision & Accuracy: SUMMARY

*To*: mathgroup at christensen.cybernetics.net*Subject*: [mg743] Precision & Accuracy: SUMMARY*From*: Paul E Howland <PEHOWLAND at taz.dra.hmg.gb>*Date*: Wed, 12 Apr 1995 15:43:49 +0200 (IST)

Mathgroup, I posted a question yesterday regarding the difference between Accuracy[] and Precision[] in Mathematica. I had two different but interesting replies, which I summarise below. To solve my problem, though, I eventually delved into the package NonlinearFit.m to see how the Pro's did it: I'll also mention my findings about this here. The first answer provided an analogy with playing darts. Accuracy implies that the average position of the darts is near the bull's eye, although their variance can be quite high, whereas Precision implies that all the darts hit the same place, but this may be nowhere near the bull's eye. That is, Accuracy=Unbiased answer, Precision=Low Variance answer. The second answer provided a useful example of where the Accuracy and Precision of a number vary: Accuracy[1.00000000000000000001 * 10^-5] = 25 Precision[1.00000000000000000001 * 10^-5] = 20 I was still left wondering how this could be applied when optimising a function. What is the difference between the accuracy of my optimisation and its precision? I had a delve into the code of NonlinearFit.m, and found the answer. When trying to minimise a cost function (NonlinearFit uses the Chi squared function), the function satisfies the accuracy constraint if: 0 <= chisq-newchi < (10^-AccuracyGoal) but satisfies the precision constraint if: 0 <= chisq-newchi < newchi*(10^-PrecisionGoal) where chisq is the value of the cost function at the last iteration, and newchi is the value of the cost function at the current iteration. Thus AccuracyGoal specifies that the algorithm be declared converged when the _absolute_ difference in the cost function is less than a certain threshold, whereas PrecisionGoal specifies that the algorithm be declared converged when the _proportional_ change in the cost function is less than a specified threshold. *** TWO WARNINGS ABOUT NonlinearFit.m *** ========================================= 1. Whilst looking at the code for NonlinearFit, I noticed that the default values for AccuracyGoal and PrecisionGoal are: AccuracyGoal -> 1 PrecisionGoal -> 3 rather than AccuracyGoal -> $MachinePrecision-10 PrecisionGoal -> $MachinePrecision-10 as stated in the "Guide to Standard Mathematica Packages V2.2". Thus results may be considerably less precise than you expect! 2. Be wary of the message "NonlinearFit::lmpnocon" which warns that the values of the parameters do not appear to have converged. The algorithm produces this warning if the difference between the previous value of a parameter and its new value is greater than 1 (for any of the parameters), at the end of the calculation. It will thus be _highly_ dependent on the units of the parameters you are fitting! Be warned! Paul E Howland Long Range Ground Radar Systems Section tel. +44 (0)1684 895767 CSS2 Division, Room BY209 fax. +44 (0)1684 896315 Defence Research Agency email: PEHOWLAND at DRA.HMG.GB Malvern, Worcs, WR14 3PS, UK. -----------------------------------------------------------------------------