MathGroup Archive 2010

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Precision of calculations


They start off that way

data = {1.0, 1000.0, Pi // N};

Precision /@ data

{MachinePrecision,MachinePrecision,MachinePrecision}

Precision /@ Gamma[data]

{MachinePrecision,12.1834,MachinePrecision}


Bob Hanlon

---- Jim Lambaugh <lambaugh at gmail.com> wrote: 

=============
Hi guys

When I write numbers in my code, I do it as (e.g.)

1.0
1000.0
Pi //N
etc...

Is it correct to say that my calculations are done with machine-
precision, i.e. with rougly 16 digit precision?

Regards,
Jim.



  • Prev by Date: Re: Why Return[] does not work?
  • Next by Date: Re: BinCounts to InterpolatingFunction
  • Previous by thread: Re: Precision of calculations
  • Next by thread: Re: Precision of calculations