       Re: Base 10 to Base 2 conversion error ?

• To: mathgroup at christensen.cybernetics.net
• Subject: [mg1787] Re: Base 10 to Base 2 conversion error ?
• From: Joe Gwinn <gwinn at sud2.ed.ray.com>
• Date: Sun, 30 Jul 1995 21:38:19 -0400
• Organization: Raytheon

```I see that I had lost sight of the original question.  The fundamental
question is mis-pointed, if I understand the underlying issue: can we use
floating point to handle money calculations in a POS terminal.

The issue comes down to the ratio of largest value to be handled to one
penny.  If the base2 logarithm of this ratio (the number of bits required
to express the ratio) is less than the width of the mantissa field by 5 or
10 bits, then floating point will do an essentially perfect job, as there
will be 5 or 10 "guard bits" to absorb roundoff and other numerical
errors.  If the ratio just fits, you will see all manner of odd behavior
in the pennies digit, when large numbers are being handled.

For example, lets assume an extreme case, a Mercedes dealer selling cars
like boxes of soap.  Assume that the car costs \$100,000.00.  Then, the
ratio is 100000/0.01= 10^7.  Log2(10^7)= 23.253 = 24 bits.  Now, an IEEE
32-bit floating-point value has a 24-bit mantissa (including the hidden
bit), so this just fits, and so won't quite work.  However, an IEEE 64-bit
float has a 53-bit mantissa (including the hidden bit), so we will have
53-24= 29 guard bits.  The arithmetic will be essentially perfect, and
certainly will be no worse than the typical packed-binary machines

Now, use of floating-point rather than scaled binary saves *lots* of
programming and debugging effort, at the expense of speed.  However, a POS
only needs to outrun a human, so speed isn't really the issue.  Generating
the displays and running the comms probably takes more time anyway.

Joe Gwinn

```

• Prev by Date: Re: Printing
• Next by Date: Re: Options in self-defined functions
• Previous by thread: Re: Base 10 to Base 2 conversion error ?
• Next by thread: 3D Axes Label & bounding box of graphics [help]