Student Support Forum: 'Precision in small number subtraction' topicStudent Support Forum > General > "Precision in small number subtraction"

 < Previous Comment | Next Comment > Help | Reply To Comment | Reply To Topic
 Author Comment/Response yehuda 04/21/13 10:45am Hi, Notice that 5.8*10^-11 is already in machine precision which is lower than 20 and one cannot increase precision Precision[5.8*10^-11] returns \$MachinePrecision the same as Precision[N[5.8*10^-11,20]] you can use 1-N[58*10^-12,20] or 1 - 5.8*10^-11`20 The first uses infinite precision for the number and let N[...,20] to set the precision to 20 the last defines the precision immediately using the ` notation yehuda URL: benshimo@bgu.ac.il,

 Subject (listing for 'Precision in small number subtraction') Author Date Posted Precision in small number subtraction avikarto 04/19/13 11:07pm Re: Precision in small number subtraction yehuda 04/21/13 10:45am Re: Precision in small number subtraction jf 04/21/13 8:54pm
 < Previous Comment | Next Comment > Help | Reply To Comment | Reply To Topic