MathGroup Archive 1995

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Base 10 to Base 2 conversion error ?

  • To: mathgroup at
  • Subject: [mg1787] Re: Base 10 to Base 2 conversion error ?
  • From: Joe Gwinn <gwinn at>
  • Date: Sun, 30 Jul 1995 21:38:19 -0400
  • Organization: Raytheon

I see that I had lost sight of the original question.  The fundamental 
question is mis-pointed, if I understand the underlying issue: can we use 
floating point to handle money calculations in a POS terminal.  

The issue comes down to the ratio of largest value to be handled to one 
penny.  If the base2 logarithm of this ratio (the number of bits required 
to express the ratio) is less than the width of the mantissa field by 5 or 
10 bits, then floating point will do an essentially perfect job, as there 
will be 5 or 10 "guard bits" to absorb roundoff and other numerical 
errors.  If the ratio just fits, you will see all manner of odd behavior 
in the pennies digit, when large numbers are being handled.

For example, lets assume an extreme case, a Mercedes dealer selling cars 
like boxes of soap.  Assume that the car costs $100,000.00.  Then, the 
ratio is 100000/0.01= 10^7.  Log2(10^7)= 23.253 = 24 bits.  Now, an IEEE  
32-bit floating-point value has a 24-bit mantissa (including the hidden 
bit), so this just fits, and so won't quite work.  However, an IEEE 64-bit 
float has a 53-bit mantissa (including the hidden bit), so we will have 
53-24= 29 guard bits.  The arithmetic will be essentially perfect, and 
certainly will be no worse than the typical packed-binary machines 
traditionally used for COBOL business data processing.

Now, use of floating-point rather than scaled binary saves *lots* of 
programming and debugging effort, at the expense of speed.  However, a POS 
only needs to outrun a human, so speed isn't really the issue.  Generating 
the displays and running the comms probably takes more time anyway.

Joe Gwinn

  • Prev by Date: Re: Printing
  • Next by Date: Re: Options in self-defined functions
  • Previous by thread: Re: Base 10 to Base 2 conversion error ?
  • Next by thread: 3D Axes Label & bounding box of graphics [help]