Mathematica 9 is now available
Student Support Forum
Student Support Forum: 'Revised accuracy question' topicStudent Support Forum > General > "Revised accuracy question"

Next Comment >Help | Reply To Topic
Author Comment/Response
david silverman
11/22/98 09:31am

Since I have to live with the limits of Mathematica, let me expand the
problem I'm having with the accuracy.

I have a latitude or longitude in a data file downloaded from NOAA. The data is
in the form DDD.MMSS where DDD is the degress, MM is the minutes, SS are the
seconds. If you are unfamiliar with this angular measure notation, a degree
is divided into 60 minutes, and a minute into 60 seconds. The data is only given
to as many digits as necessary, so if the data is exactly 40 degrees, the input
file only has 40, if the data is 40 degrees, 30 seconds, the input file only
gives 40.3.

I need to take this number which I have no control over, and convert it to a
number that is in degrees only. For instance, 40 degrees, 30 seconds is
40.5 degrees.

I tried to do the conversion including SetAccuracy, but I still cannot get the
correct results by using:
IntegerPart[a] +
IntegerPart[100 FractionalPart[SetAccuracy[a,3]]]/60
+FractionalPart[100 SetAccuracy[a,5]]/36

If I give 44.3 as input, I should get 44.5000 as output, but I get 44.5111.
Other numbers work OK. 44.2 gives 44.3333333333333; 44.6 gives 45.000000000000;
44.596 gives 45.0000000000000; 45.296 gives 46.5000000000000.
What would be the best way to correct this?



URL: ,

Subject (listing for 'Revised accuracy question')
Author Date Posted
Revised accuracy question david silver... 11/22/98 09:31am
Re: Revised accuracy question dave 11/25/98 10:20am
Next Comment >Help | Reply To Topic