Re: Base 10 to Base 2 conversion error ?
- To: mathgroup at christensen.cybernetics.net
- Subject: [mg1787] Re: Base 10 to Base 2 conversion error ?
- From: Joe Gwinn <gwinn at sud2.ed.ray.com>
- Date: Sun, 30 Jul 1995 21:38:19 -0400
- Organization: Raytheon
I see that I had lost sight of the original question. The fundamental question is mis-pointed, if I understand the underlying issue: can we use floating point to handle money calculations in a POS terminal. The issue comes down to the ratio of largest value to be handled to one penny. If the base2 logarithm of this ratio (the number of bits required to express the ratio) is less than the width of the mantissa field by 5 or 10 bits, then floating point will do an essentially perfect job, as there will be 5 or 10 "guard bits" to absorb roundoff and other numerical errors. If the ratio just fits, you will see all manner of odd behavior in the pennies digit, when large numbers are being handled. For example, lets assume an extreme case, a Mercedes dealer selling cars like boxes of soap. Assume that the car costs $100,000.00. Then, the ratio is 100000/0.01= 10^7. Log2(10^7)= 23.253 = 24 bits. Now, an IEEE 32-bit floating-point value has a 24-bit mantissa (including the hidden bit), so this just fits, and so won't quite work. However, an IEEE 64-bit float has a 53-bit mantissa (including the hidden bit), so we will have 53-24= 29 guard bits. The arithmetic will be essentially perfect, and certainly will be no worse than the typical packed-binary machines traditionally used for COBOL business data processing. Now, use of floating-point rather than scaled binary saves *lots* of programming and debugging effort, at the expense of speed. However, a POS only needs to outrun a human, so speed isn't really the issue. Generating the displays and running the comms probably takes more time anyway. Joe Gwinn