Re: Counting number of numbers in a large list between two valus
- To: mathgroup at smc.vnet.net
- Subject: [mg114548] Re: Counting number of numbers in a large list between two valus
- From: truxton spangler <truxtonspangler29 at gmail.com>
- Date: Tue, 7 Dec 2010 06:47:36 -0500 (EST)
- References: <idigg5$iq2$1@smc.vnet.net>
On Dec 6, 10:13 pm, "Carl K. Woll" <ca... at wolfram.com> wrote:
> On 12/5/2010 8:57 PM, Lyle wrote:
>
>
>
> > Dear Listers,
>
> > I have a large (5-20million) one dimensional list of real numbers and
> > I want to count the number of entries in the list that lie between 2
> > specific values (x1, x2). I need to run the function for a number of
> > different ranges.
>
> > ie. number of list entries (l), where x1<= l<= x2
>
> > I've tried:
>
> > tallydata[{x1_, x2_}] := Count[data, x_ /; x1<= x<= x2]
>
> > that takes about 3-4 seconds
>
> > and
>
> > tallydata[{x1_, x2_}] := Length[Select[data, x1<= #<= x2&]]
>
> > which takes a little bit longer.
>
> > The best I've managed is (this last one might be off by 1 or 2 but
> > this doesn't really matter to me):
>
> > sorteddata = Sort[data];
> > nf = Nearest[sorteddata];
> > tallyrange[{x1_, x2_}] :=
> > First[Position[sorteddata, First[nf[x2]]]] -
> > First[Position[sorteddata, First[nf[x1]]]]
>
> > which takes between 1 and 2 seconds but I was hoping there might be a
> > faster way to do this?
>
> > Any help would be great!
>
> > Thanks,
> > Lyle Gordon
>
> > Northwestern University
>
> Here's one possibility:
>
> data = RandomReal[1, 10^7];
>
> In[206]:= Total@Unitize@Clip[data, {.2, .3}, {0, 0}] // Timing
>
> Out[206]= {0.14, 1000695}
>
> Carl Woll
> Wolfram Research
Why is it that a function called Count is not the fastest way to
count, a function called BinCount is not the fastest way to count
bins, a function called Sort is rarely the fastest way to sort, a
function called Select is rarely the fastest way to select ...and so
on. When "named" built in functions are rarely the best programming
option this seems like a flaw in the design philosophy of the software
and certainly is an added barrier to Mathematica becoming an intuitive program
to learn.
While I am on this topic the date and time handling functions in Mathematica,
even in version 8, are so slow as to be useless for real work. This is
another example of named functions not being the best programming
option. Ditto the time value of money functions. It is very easy to
get 2 orders of magnitude speed enhancement writing your own code
which brings into question the point of developing these sort of built
in functions in the form that they currently exist.