Re: Working with Indeterminate in large numerical lists
- To: mathgroup at smc.vnet.net
- Subject: [mg100242] Re: Working with Indeterminate in large numerical lists
- From: pfalloon <pfalloon at gmail.com>
- Date: Fri, 29 May 2009 20:56:43 -0400 (EDT)
- References: <gvlhu1$dos$1@smc.vnet.net> <gvn73a$et4$1@smc.vnet.net>
On May 29, 9:35 am, dh <d... at metrohm.com> wrote: > Hi, > > simply try negation: > > Select[x, ! NumberQ[#] &] > > Daniel > > > > pfalloon wrote: > > Hi everyone, > > I'm wondering about the optimal way to work with Indeterminate in > > large matrices. I've been using this to replace "bad" data points that > > I want to prevent from polluting calculations involving lists of data, > > but I'm not sure I'm working as smartly as I could be. > > > As an example, suppose I have a list of machine-precision reals and > > some Indeterminate elements: > > > x = RandomSample[Join[RandomReal[1, 1000], ConstantArray > > [Indeterminate, 10]]; > > > If I want to take only the valid entries, the best I have been able to > > find is something like: > > > Select[x, NumberQ] > > > This seems to work reasonably well. But if I want to specifically > > select the Indeterminate entries, there doesn't seem to be any > > function (equivalent to, say, "isnan" from another system), so I have t= o > > resort to something less succinct like > > > Select[x, # === Indeterminate &] > > > Does anyone have any suggestions on better ways to do this, or any > > general tips for working with Indeterminate in this context? > > > I'm particularly keen to do things in the most efficient way possible > > as I'm working with rather large lists. > > > Thanks, > > Peter. Hi again, Thanks all for the helpful suggestions. Playing around with these, it seems like the neatest solution (in terms of elegance + performance) for my purposes is to use DeleteCases[lst, Indeterminate] when working with a list. A related task is to remove rows of a matrix which contain one or more Indeterminate entry. In this case the following seem to be optimal: Select[mat, FreeQ[#, Indeterminate] &] (* to select valid numeric rows *) and either of Select[mat, MemberQ[#, Indeterminate] &] DeleteCases[mat, {___, Indeterminate, ___}] (* to select invalid rows containing one or more Indeterminate *) I'm slightly surprised that the latter pattern-matching approach is as efficient as the previous one, but it seems to be the case. Thanks again, Peter.