MathGroup Archive 2009

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Working with Indeterminate in large numerical lists

  • To: mathgroup at smc.vnet.net
  • Subject: [mg100227] Re: [mg100207] Working with Indeterminate in large numerical lists
  • From: Bob Hanlon <hanlonr at cox.net>
  • Date: Thu, 28 May 2009 19:33:17 -0400 (EDT)
  • Reply-to: hanlonr at cox.net

$HistoryLength = 0;

n = 1000;

x = RandomSample[Join[RandomReal[1, 100 n],
    ConstantArray[Indeterminate, n]]];

Timing[a1 = Select[x, ! NumberQ[#] &];]

{0.096885,Null}

However, note that

NumberQ /@ {1, 1., Pi, E, I}

{True,True,False,False,True}

NumericQ /@ {1, 1., Pi, E, I}

{True,True,True,True,True}

So in most cases, you would want to use NumericQ rather than NumberQ

Timing[a2 = Select[x, ! NumericQ[#] &];]

{0.096093,Null}

Timing[a3 = Select[x, # === Indeterminate &];]

{0.069981,Null}

Oddly, this is faster

Timing[a4 = x[[Flatten[Position[x, Indeterminate]]]];]

{0.016217,Null}

Cases appears to be much more efficient than Select in this case

Timing[a5 = Cases[x, Indeterminate];]

{0.007767,Null}

a1 === a2 === a3 === a4 ===
 a5 === ConstantArray[Indeterminate, n]

True

However, for valid entries

Timing[b1 = Select[x, NumericQ];]

{0.035489,Null}

Timing[b2 = Cases[x, _?NumericQ];]

{0.064408,Null}

b1 === b2

True


Bob Hanlon

---- pfalloon <pfalloon at gmail.com> wrote: 

=============
Hi everyone,
I'm wondering about the optimal way to work with Indeterminate in
large matrices. I've been using this to replace "bad" data points that
I want to prevent from polluting calculations involving lists of data,
but I'm not sure I'm working as smartly as I could be.

As an example, suppose I have a list of machine-precision reals and
some Indeterminate elements:

x = RandomSample[Join[RandomReal[1, 1000], ConstantArray
[Indeterminate, 10]];

If I want to take only the valid entries, the best I have been able to
find is something like:

Select[x, NumberQ]

This seems to work reasonably well. But if I want to specifically
select the Indeterminate entries, there doesn't seem to be any
function (equivalent to, say, "isnan" from another system), so I have to
resort to something less succinct like

Select[x, #  === Indeterminate &]

Does anyone have any suggestions on better ways to do this, or any
general tips for working with Indeterminate in this context?

I'm particularly keen to do things in the most efficient way possible
as I'm working with rather large lists.

Thanks,
Peter.




  • Prev by Date: Re: Re: new to group - Chemical Equation
  • Next by Date: Re: NDSolve with Numeric Function
  • Previous by thread: Re: Working with Indeterminate in large numerical lists
  • Next by thread: Re: Working with Indeterminate in large numerical lists