Re: Working with Indeterminate in large numerical lists
- To: mathgroup at smc.vnet.net
- Subject: [mg100242] Re: Working with Indeterminate in large numerical lists
- From: pfalloon <pfalloon at gmail.com>
- Date: Fri, 29 May 2009 20:56:43 -0400 (EDT)
- References: <gvlhu1$dos$1@smc.vnet.net> <gvn73a$et4$1@smc.vnet.net>
On May 29, 9:35 am, dh <d... at metrohm.com> wrote:
> Hi,
>
> simply try negation:
>
> Select[x, ! NumberQ[#] &]
>
> Daniel
>
>
>
> pfalloon wrote:
> > Hi everyone,
> > I'm wondering about the optimal way to work with Indeterminate in
> > large matrices. I've been using this to replace "bad" data points that
> > I want to prevent from polluting calculations involving lists of data,
> > but I'm not sure I'm working as smartly as I could be.
>
> > As an example, suppose I have a list of machine-precision reals and
> > some Indeterminate elements:
>
> > x = RandomSample[Join[RandomReal[1, 1000], ConstantArray
> > [Indeterminate, 10]];
>
> > If I want to take only the valid entries, the best I have been able to
> > find is something like:
>
> > Select[x, NumberQ]
>
> > This seems to work reasonably well. But if I want to specifically
> > select the Indeterminate entries, there doesn't seem to be any
> > function (equivalent to, say, "isnan" from another system), so I have t=
o
> > resort to something less succinct like
>
> > Select[x, # === Indeterminate &]
>
> > Does anyone have any suggestions on better ways to do this, or any
> > general tips for working with Indeterminate in this context?
>
> > I'm particularly keen to do things in the most efficient way possible
> > as I'm working with rather large lists.
>
> > Thanks,
> > Peter.
Hi again,
Thanks all for the helpful suggestions. Playing around with these, it
seems like the neatest solution (in terms of elegance + performance)
for my purposes is to use
DeleteCases[lst, Indeterminate]
when working with a list.
A related task is to remove rows of a matrix which contain one or more
Indeterminate entry. In this case the following seem to be optimal:
Select[mat, FreeQ[#, Indeterminate] &] (* to select valid numeric rows
*)
and either of
Select[mat, MemberQ[#, Indeterminate] &]
DeleteCases[mat, {___, Indeterminate, ___}] (* to select invalid rows
containing one or more Indeterminate *)
I'm slightly surprised that the latter pattern-matching approach is as
efficient as the previous one, but it seems to be the case.
Thanks again,
Peter.