| Author |
Comment/Response |
Rick Bischoff
|
 02/11/02 06:42am
Hi,
I was evaluating
(256^3 - 1)
-----------
256^2
Mathematica gives the correct fraction for this:
16777215
--------
65536
But, when I issue the N[%] command to get the decimal approximation, it incorrectly returns
256
Instead of 255.99....
I tried N[%, 10] thru N[%,20] and finally got it to show the right answer at 20... Is there any to set the default precision on N[%] operations (and other functions that return real numbers, etc) to 20+ so I don't get confused anymore??
Thanks,
Rick
URL: , |
|
