Services & Resources / Wolfram Forums / MathGroup Archive
-----

MathGroup Archive 2008

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: A strange precision behavior?

  • To: mathgroup at smc.vnet.net
  • Subject: [mg90404] Re: A strange precision behavior?
  • From: dh <dh at metrohm.ch>
  • Date: Tue, 8 Jul 2008 02:28:31 -0400 (EDT)
  • References: <g4smba$2g$1@smc.vnet.net>


Hi,

0.01 is MachinePrecision. If you bloat the accuracy of some numbe, 

random (binary) digits are added, that is the content of the 

corresponding memory.

hope this helps, Daniel



Aaron Fude wrote:

> The following two commands result in different values:

> 

> SetPrecision[0.01, 50]

> 0.01`50

> 

> Is that because in the first case, 0.01 is first represented as a

> double and then passed to SetPrecision, while in the second case,

> Mathematica gets a chance to interprete 0.01 as 1/100?

> 

> Thanks!

> 

> Aaron

> 





-- 



Daniel Huber

Metrohm Ltd.

Oberdorfstr. 68

CH-9100 Herisau

Tel. +41 71 353 8585, Fax +41 71 353 8907

E-Mail:<mailto:dh at metrohm.com>

Internet:<http://www.metrohm.com>




  • Prev by Date: Trouble related to "CSV"
  • Next by Date: Extracting points from a contour plot
  • Previous by thread: A strange precision behavior?
  • Next by thread: Re: A strange precision behavior?