       Re: Re: Correlation function and data

• To: mathgroup at smc.vnet.net
• Subject: [mg24785] Re: [mg24748] Re: Correlation function and data
• From: Daniel Lichtblau <danl at wolfram.com>
• Date: Thu, 10 Aug 2000 00:32:26 -0400 (EDT)
• References: <200008090631.CAA00912@smc.vnet.net>
• Sender: owner-wri-mathgroup at wolfram.com

```Deborah Leddon wrote:
>
> Hello,
>  I am trying to construct an autocorrelation function that can be
> applied to a data set as follows;
>
> n=Length[data];
> c[y_]:= Sum[(data[[i]] - Mean[data] )* ( data[[i+y]] - Mean[data])/
>                 Variance[data], {i,1,n - y}];
> corrfunction = Array[c, {n - 1}];
>
> ListPlot[corrfunction, PlotJoined->True]
>
> Problem is , this routine works well for data sets up to a length of
> 300 points, but gets unusually long for larger data sets , say
> around 1000-3000 points in length. I've tried  "Map" (using
> anonymous function rules), Table-Evaluate, etc..
>
> Anyone got any ideas? I would much appreciate them!
>
> Regards,
> Deb L.

You are doing alot of recomputation. If you precompute the mean and
variance, and translate the data before doing the correlations, you will
get a factor of n improvement.

<<Statistics`

data = Table[Random[], {1000}];
n = Length[data];
mean = Mean[data];
transdata = data - mean;
var = Variance[data];
c[y_]:= Sum[transdata[[i]]*transdata[[i+y]], {i,1,n-y}] / var;

In:= Timing[corrfunction = Array[c, {n-1}];]
Out= {20.64 Second, Null}

You can make this considerably faster using ListCorrelate instead of
Sum.

In:= Timing[c2 = ListCorrelate[transdata, Drop[transdata,1], 1, 0] /
var;]
Out= {0.01 Second, Null}

In:= Max[Abs[c2-corrfunction]]
-13
Out= 1.42109 10

Daniel Lichtblau
Wolfram Research

```

• Prev by Date: Re: Help: 1.#QNAN ??
• Next by Date: Re: 2nd arg to BeginPackage
• Previous by thread: Re: Correlation function and data
• Next by thread: Re: Re: Correlation function and data