Services & ResourcesWolfram ForumsMathGroup Archive

[Date Index] [Thread Index] [Author Index]

Empirical CDF and InterpolatingFunction

• To: mathgroup at smc.vnet.net
• Subject: [mg36555] Empirical CDF and InterpolatingFunction
• From: mark at markfisher.net (Mark Fisher)
• Date: Wed, 11 Sep 2002 13:27:40 -0400 (EDT)
• Sender: owner-wri-mathgroup at wolfram.com

I'm trying to write a fast empirical cummulative distribution function
(CDF). Empirical CDFs are step functions that can be expressed in
terms of a Which statement. For example, given the list of
observations {1, 2, 3},

f = Which[# < 1, 0, # < 2, 1/3, # < 3, 2/3, True, 1]&

is the empirical CDF. Note that f /@ {1, 2, 3} returns {1/3, 2/3, 1}
and f is continuous from the right.

When the number of observations is large, the Which statement
evaluates fairly slowly (even if it has been Compiled). Since
InterpolationFunction evaluates so much faster in general, I've tried
to use Interpolation with InterpolationOrder -> 0. The problem is that
the resulting InterpolatingFunction doesn't behave the way (I think)
it ought to. For example, let

g = Interpolation[{{1, 1/3}, {2, 2/3}, {3, 1}}, InterpolationOrder ->
0]

Then, g /@ {1, 2, 3} returns {2/3, 2/3, 1} instead of {1/3, 2/3, 1}.
In addition, g is continuous from the left rather than from the right.

Obviously I am not aware of the considerations that went into
determining the behavior of InterpolationFunction when
InterpolationOrder -> 0.

So I have two questions: 

(1) Does anyone have any opinions about how InterpolatingFunction
ought to behave with InterpolationOrder -> 0?

(2) Does anyone have a faster way to evaluate an empirical CDF than a
compiled Which function?

By the way, here's my current version:

CompileEmpiricalCDF[list_?(VectorQ[#, NumericQ] &)] :=
  Block[{x}, Compile[{{x, _Real}}, Evaluate[
    Which @@ Flatten[
      Append[
          Transpose[{
            Thread[x < Sort[list]],
            Range[0, 1 - 1/#, 1/#] & @ Length[list]
              }],
        {True, 1}]]
  ]]]

--Mark

• Follow-Ups:
  • Re: Empirical CDF and InterpolatingFunction
    • From: Tomas Garza <tgarza01@prodigy.net.mx>