Mathematica 9 is now available
Student Support Forum
-----
Student Support Forum: 'Precision in small number subtraction' topicStudent Support Forum > General > Archives > "Precision in small number subtraction"

Next Comment >Help | Reply To Topic
Author Comment/Response
avikarto
04/19/13 11:07pm

I'm trying to subtract small numbers from one, and not getting a correct result.

In[1]:= 1-(5.8*10^-11)
Out[1]= 1


I tried changing the notebook's $MinPrecision to 20, and tried

In[2]:= 1-N[5.8*10^-11,20]
Out[2]= 1

which was obviously no help.

Strangely,

In[401]:= N[E^-40, 20]
Out[401]= 4.2483542552915889953*10^-18

In[402]:= 1 - %
Out[402]= 0.99999999999999999575164574470841100467

seems to work, but not the simple subtraction that I need to do. Any help would be appreciated. Thanks!

URL: ,

Subject (listing for 'Precision in small number subtraction')
Author Date Posted
Precision in small number subtraction avikarto 04/19/13 11:07pm
Re: Precision in small number subtraction yehuda 04/21/13 10:45am
Re: Precision in small number subtraction jf 04/21/13 8:54pm
Next Comment >Help | Reply To Topic