[Date Index] [Thread Index] [Author Index]
Re: Decimal math bug?
Hi, Paul! Paul Lutus wrote: > [...] > > > In:=(.53+.9) - (1.43) == 0 > > Out:=False > > Second, your example is wrong. 53 + 9 = 62, not 143. 53 + 9 - 62 == 0 > always, not sometimes. > So I think 53 + 9 = 62, but 0.53 + 0.9 = 1.43. If you multiply the last equation with 100, you will have 53 + 90 = 143 In the thread i agree with you, that the numerical precision is not the algebraic precision. The issue to solve this problem is called "condition". You must calculated the condition of the equation, before you will have "good" results. Example: ie the numerical precision is called eps = 1E-3 So you have 0.53+eps = 0.53 The equation results in: (0.53+eps) - (0.9 - eps) == (1.43+eps) or every permutation of sign of eps. You have many results. The one, you will get, is based on the implemantation of your operation system or the numerical coprocessor. CU Christian Böhm