Re: Derivative of experimental data
- To: mathgroup at smc.vnet.net
- Subject: [mg124712] Re: Derivative of experimental data
- From: Yves Klett <yves.klett at googlemail.com>
- Date: Thu, 2 Feb 2012 04:53:29 -0500 (EST)
- Delivered-to: l-mathgroup@mail-archive0.wolfram.com
- References: <jgauad$eq8$1@smc.vnet.net>
Hi, The HPFilter works pretty well for some of my smoothing cases, too. Probalby not helpful, but did you try supplying your own smoothing parameter to the HPFilter function? At some version switch I had problems with the automatically chosen parameter so I started supplying my own values. The algorithm responds rather benignly to roughly chosen (trial and error) values. Another thing would be to try your hand at a frequency-based filter (e.g. using FourierDCT and such). Two commercial applications also come to mind, the EDA package (http://www.wolfram.com/products/applications/eda/) and (possibly completely O.T.T. for casual use) DataModeler by Evolved Analytics. I'd like to see some more state-of-the-art smoothing techniques implemented into the standard functionality of Mathematica. Regards, Yves Am 01.02.2012 09:49, schrieb Gabriel Landi: > Dear MathGroup users, > > I have a question which is very important for my current research, and > which involves not only Mathematica, but computer science in general. > > I have experimental data which is not very noisy, a small example of > which may be downloaded here. > > Basically I need to compute the derivative of this data. Here are my > options so far: > > Option 1: Finite differencing. Its terrible since the noise enhances > dramatically. > Option 2: Fitting some arbitrary function. The problem is that the > general functional form of the data changes from experiment to > experiment, so it is not possible to find a function which fits > adequately in all cases. > Option 3: Savitzky-Golay filters (self-implemented in Mathematica, based > on the discussion in Numerical Recipes, 3rd Ed.). It doesn't seem to > make much of a difference; probably because my data is not really that > noisy. > Option 4: Smoothing Splines filter. I am currently using Mr. Ludsteck > package HPFilter. So far it is by far the best outcome. However, it is > not free of some wild oscillations that are clearly non-analytical and > which are giving me quite the headache. > > Any suggestions are more than welcome. > I really appreciate any help I can get. > > Best regards, > > Gabriel Landi