Student Support Forum: 'Revised accuracy question' topicStudent Support Forum > General > "Revised accuracy question"

 Next Comment > Help | Reply To Topic
 Author Comment/Response david silverman 11/22/98 09:31am Since I have to live with the limits of Mathematica, let me expand the problem I'm having with the accuracy. I have a latitude or longitude in a data file downloaded from NOAA. The data is in the form DDD.MMSS where DDD is the degress, MM is the minutes, SS are the seconds. If you are unfamiliar with this angular measure notation, a degree is divided into 60 minutes, and a minute into 60 seconds. The data is only given to as many digits as necessary, so if the data is exactly 40 degrees, the input file only has 40, if the data is 40 degrees, 30 seconds, the input file only gives 40.3. I need to take this number which I have no control over, and convert it to a number that is in degrees only. For instance, 40 degrees, 30 seconds is 40.5 degrees. I tried to do the conversion including SetAccuracy, but I still cannot get the correct results by using: IntegerPart[a] + IntegerPart[100 FractionalPart[SetAccuracy[a,3]]]/60 +FractionalPart[100 SetAccuracy[a,5]]/36 If I give 44.3 as input, I should get 44.5000 as output, but I get 44.5111. Other numbers work OK. 44.2 gives 44.3333333333333; 44.6 gives 45.000000000000; 44.596 gives 45.0000000000000; 45.296 gives 46.5000000000000. What would be the best way to correct this? Thanks, David URL: ,

 Subject (listing for 'Revised accuracy question') Author Date Posted Revised accuracy question david silver... 11/22/98 09:31am Re: Revised accuracy question dave 11/25/98 10:20am
 Next Comment > Help | Reply To Topic