Re: floating point issue
- To: mathgroup at smc.vnet.net
- Subject: [mg86808] Re: [mg86748] floating point issue
- From: Bob Hanlon <hanlonr at cox.net>
- Date: Fri, 21 Mar 2008 01:56:14 -0500 (EST)
- Reply-to: hanlonr at cox.net
K = Rationalize[{111.5, 10.5, 1.5}, 0];
g = Rationalize[{-0.7071068, 0., -0.7071068}, 0];
K.Cross[K, g]
0
K = {111.5`25, 10.5`25, 1.5`25};
g = {-0.7071068`25, 0.`25, -0.7071068`25};
Chop[K.Cross[K, g], 10^-18]
0
Bob Hanlon
---- Chris Scullard <scullard at uchicago.edu> wrote:
> Hi everyone,
>
> I wonder if I can get some opinions on the best way to deal with this
> precision issue I am having. I define the vectors:
>
> K = {111.5, 10.5, 1.5}
> g={-0.7071068, 0., -0.7071068}
>
> And I need this:
>
> K.Cross[K, g]
>
> to be 0 in accordance with a vector identity. The answer comes out to
> around 1.3 x 10^(-13), which is certainly close to 0 but not close
> enough for what I'm doing. I've tried various things like writing out
> the cross product explicitly without using the functions but the result
> is the same. And using N in various places doesn't seem to help either.
> What's the standard solution for this kind of thing?
>
> Thanks,
> Chris
>