Re: Working with Indeterminate in large numerical lists
- To: mathgroup at smc.vnet.net
- Subject: [mg100227] Re: [mg100207] Working with Indeterminate in large numerical lists
- From: Bob Hanlon <hanlonr at cox.net>
- Date: Thu, 28 May 2009 19:33:17 -0400 (EDT)
- Reply-to: hanlonr at cox.net
$HistoryLength = 0;
n = 1000;
x = RandomSample[Join[RandomReal[1, 100 n],
ConstantArray[Indeterminate, n]]];
Timing[a1 = Select[x, ! NumberQ[#] &];]
{0.096885,Null}
However, note that
NumberQ /@ {1, 1., Pi, E, I}
{True,True,False,False,True}
NumericQ /@ {1, 1., Pi, E, I}
{True,True,True,True,True}
So in most cases, you would want to use NumericQ rather than NumberQ
Timing[a2 = Select[x, ! NumericQ[#] &];]
{0.096093,Null}
Timing[a3 = Select[x, # === Indeterminate &];]
{0.069981,Null}
Oddly, this is faster
Timing[a4 = x[[Flatten[Position[x, Indeterminate]]]];]
{0.016217,Null}
Cases appears to be much more efficient than Select in this case
Timing[a5 = Cases[x, Indeterminate];]
{0.007767,Null}
a1 === a2 === a3 === a4 ===
a5 === ConstantArray[Indeterminate, n]
True
However, for valid entries
Timing[b1 = Select[x, NumericQ];]
{0.035489,Null}
Timing[b2 = Cases[x, _?NumericQ];]
{0.064408,Null}
b1 === b2
True
Bob Hanlon
---- pfalloon <pfalloon at gmail.com> wrote:
=============
Hi everyone,
I'm wondering about the optimal way to work with Indeterminate in
large matrices. I've been using this to replace "bad" data points that
I want to prevent from polluting calculations involving lists of data,
but I'm not sure I'm working as smartly as I could be.
As an example, suppose I have a list of machine-precision reals and
some Indeterminate elements:
x = RandomSample[Join[RandomReal[1, 1000], ConstantArray
[Indeterminate, 10]];
If I want to take only the valid entries, the best I have been able to
find is something like:
Select[x, NumberQ]
This seems to work reasonably well. But if I want to specifically
select the Indeterminate entries, there doesn't seem to be any
function (equivalent to, say, "isnan" from another system), so I have to
resort to something less succinct like
Select[x, # === Indeterminate &]
Does anyone have any suggestions on better ways to do this, or any
general tips for working with Indeterminate in this context?
I'm particularly keen to do things in the most efficient way possible
as I'm working with rather large lists.
Thanks,
Peter.