MathGroup Archive 2012

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Derivative of experimental data

  • To: mathgroup at
  • Subject: [mg124712] Re: Derivative of experimental data
  • From: Yves Klett <yves.klett at>
  • Date: Thu, 2 Feb 2012 04:53:29 -0500 (EST)
  • Delivered-to:
  • References: <jgauad$eq8$>


The HPFilter works pretty well for some of my smoothing cases, too.
Probalby not helpful, but did you try supplying your own smoothing
parameter to the HPFilter function? At some version switch I had
problems with the automatically chosen parameter so I started supplying
my own values. The algorithm responds rather benignly to roughly chosen
(trial and error) values.

Another thing would be to try your hand at a frequency-based filter
(e.g. using FourierDCT and such).

Two commercial applications also come to mind, the EDA package
(  and (possibly
completely O.T.T. for casual use) DataModeler by Evolved Analytics.

I'd like to see some more state-of-the-art smoothing techniques
implemented into the standard functionality of Mathematica.


Am 01.02.2012 09:49, schrieb Gabriel Landi:
> Dear MathGroup users,
> I have a question which is very important for my current research, and 
> which involves not only Mathematica, but computer science in general.
> I have experimental data which is not very noisy, a small example of 
> which may be downloaded here. 
> Basically I need to compute the derivative of this data. Here are my 
> options so far:
> Option 1: Finite differencing. Its terrible since the noise enhances 
> dramatically.
> Option 2: Fitting some arbitrary function. The problem is that the 
> general functional form of the data changes from experiment to 
> experiment, so it is not possible to find a function which fits 
> adequately in all cases.
> Option 3: Savitzky-Golay filters (self-implemented in Mathematica, based 
> on the discussion in Numerical Recipes, 3rd Ed.). It doesn't seem to 
> make much of a difference; probably because my data is not really that 
> noisy.
> Option 4: Smoothing Splines filter. I am currently using Mr. Ludsteck 
> package HPFilter. So far it is by far the best outcome. However, it is 
> not free of some wild oscillations that are clearly non-analytical and 
> which are giving me quite the headache.
> Any suggestions are more than welcome.
> I really appreciate any help I can get.
> Best regards,
> Gabriel Landi

  • Prev by Date: Re: Plotter for complex polynomials (complex coefficients)
  • Next by Date: Re: Plotter for complex polynomials (complex coefficients)
  • Previous by thread: Re: Derivative of experimental data
  • Next by thread: Re: Derivative of experimental data