Re: Noisy data and ListConvolve
- To: mathgroup at smc.vnet.net
- Subject: [mg45882] Re: Noisy data and ListConvolve
- From: Bill Rowe <readnewsciv at earthlink.net>
- Date: Wed, 28 Jan 2004 05:19:33 -0500 (EST)
- Sender: owner-wri-mathgroup at wolfram.com
On 1/27/04 at 4:51 AM, tesiramy at omrf.ouhsc.edu (Yasvir Tesiram)
wrote:
>Hi all, I recently wanted to smooth out some noisy data and compare
>it with a fitting procedure. My enquires led me to ListConvolve.
>Going through the examples provided in the help files, some
>questions arose which I hope someone will be able to answer for me
>or point me to some book or something that goes through this type
>of analysis.
>1. How does one choose an appropriate kernel?
It will depend a great deal on what you want to achieve, the nature of the data and the nature of the noise you want to filter. If you have data that is constant with additive noise that is symetric around a zero mean, then implementing a moving average with ListConvolve should be quite effective. A kernel for a moving average of length n can be generated as Table[1/n, {1, n}]
>2. I want the same number of data points as my original set of points.
>Yet, despite the documentation, even the example returns a list which
>is shorter than the original data set. How do you control this?
You control this with the third argument to ListConvolve. For example,
ListConvolve[Table[1/n, {n,4}], Table[Random[Real], {20}], {-1,-1}] will output a list of 20 numbers
There are some issues to watch for here. What is being done is to add sufficient elements to the data list at one end so that the kernel can be aligned with each of the original data elements. These additional elements are taken from the other end of the data list. That is
ListConvolve[{1,1}, {a,b,c}, {-1,-1}] yields the same result as
ListConvolve[{1,1}, {a,b,c,a}]
If you have data that is expected to be constant, moving elements from one end to the other will not be signficant. But if your data represents time varying values ordered in time, then moving data from one end to the other is likely to produce misleading plots at the list ends. But this issue of how to deal with the end of a data list (or keeping the length of the output data list the same as the input data list) is common to all data smoothing techniques I am aware of.
>3. The plot in the example looks great compared with the noisy
>generated data, but the spread of y-values after ListConvolve is
>applied is completely different.
You didn't supply details as to what example or what kernel you are talking about here so it is hard to be specific. But the idea of data smoothing is to reduce noise. Adding noise will tend to increase the spread in the data. So reducing noise should reduce the spread in the data.
>Does anyone use ListConvolve regularly for smoothing data. If so, your
>help would be greatly appreciated.
I use ListConvolve as one of several tools to be applied for data smoothing. Probably the tool I use the most often is a wavelet based technique that could be implemented with ListConvolve.
--
To reply via email subtract one hundred and four