MathGroup Archive 2005

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Table function

  • To: mathgroup at smc.vnet.net
  • Subject: [mg55008] Re: [mg54974] Table function
  • From: DrBob <drbob at bigfoot.com>
  • Date: Wed, 9 Mar 2005 06:34:33 -0500 (EST)
  • References: <200503081004.FAA23337@smc.vnet.net>
  • Reply-to: drbob at bigfoot.com
  • Sender: owner-wri-mathgroup at wolfram.com

First of all, there is no decent InputForm for error messages. I really wish there were!

Secondly, that's not a correlation matrix, at all.

Needs["Statistics`MultiDescriptiveStatistics`"]
data={{40 ,100},{50, 110},{60, 120},{70, 130}};
Transpose[data].data/Length@data
CorrelationMatrix@data

{{3150,6450},{6450,13350}}

{{1,1},{1,1}}

The first output is what you've defined, and the second is the correlation matrix.

Here's an example of normalizing the data and computing a correlation matrix manually:

normalized = Transpose[(Transpose@data - Mean@
     data)/StandardDeviationMLE@data]
Transpose[normalized].normalized/Length@normalized

{{-(3/Sqrt[5]), -(3/Sqrt[5])},
   {-(1/Sqrt[5]), -(1/Sqrt[5])},
   {1/Sqrt[5], 1/Sqrt[5]},
   {3/Sqrt[5], 3/Sqrt[5]}}

{{1,1},{1,1}}

And finally, here's (part of) a correlation matrix the size you mentioned:

data = Array[Random[] &, {246, 105}];
Short[cm = CorrelationMatrix@data, 10]

{{1., 0.00676431, 0.0543823, 0.00841345, -0.114871, 0.0242682, -0.0280324, \
0.048969, \[LeftSkeleton]90\[RightSkeleton], 0.115643, 0.0372576, -0.0276624, \
0.0606759, -0.0235359, -0.0296748, -0.133038}, \[LeftSkeleton]103\
\[RightSkeleton], {\[LeftSkeleton]1\[RightSkeleton]}}

MemoryInUse[ ]

4608000

The computation took a tenth of a second in version 5.1.1.

Bobby

On Tue, 8 Mar 2005 05:04:23 -0500 (EST), Adam Getchell <agetchell at physics.ucdavis.edu> wrote:

> Hi all,
>
> Thanks to your help, I've gotten my time series data arranged as
> year-columns of values during trading days.
>
> So now I'm calculating the correlation matrix, C_i,j = 1/(246)*[Sum over
> k=1 to 246]z_i(k)*z_j(k)
>
> where C_i,j is the individual correlation matrix, z_i(k) is the k-th row
> in the i-th column vector of data, and z_j(k) is the k-th row of the
> j-th column vector of data, e.g.
>
> 40  100
> 50  110
> 60   120
> 70   130
>
> Then C_1,1 = (1/246)*[40*40+50*50+60*60+....]
> and C_1,2 = (1/246)*[40*100+50*110+60*120+....]
>
> So yes the matrix is symmetric. I'm using the following mathematica code
> for 105 columns each with 246 rows of data:
>
> CorrelationMatrix=Table[1/246*correlationinput[[i,k]]*correlationinput[[j,k]],{i,1,105},{j,1,105},{k,1,246}]
>
> where the values are stored in the matrix correlationinput.
>
> However, the errors I get are:
>
> In[31]:=
> CorrelationMatrix =
>   Table[(1/247)*
>     correlationinput[[i,k]]*
>     correlationinput[[j,k]],
>    {i, 1, 105}, {j, 1, 105},
>    {k, 1, 246}]
>  From In[31]:=
> \!\(\*
>   RowBox[{\(Part::"partw"\),
>     ":", "\<\"Part \\!\\(106\\) of \\!\\({40.37`, 49.31`, 60.41`, 68.13`, \
> 70.44`, 64.32`, 64.6`, 47.77`, 70.39`, 95, \\(\\(\[LeftSkeleton] 95 \
> \[RightSkeleton]\\)\\)}\\) does not exist. \\!\\(\\*ButtonBox[\\\"More\
> \[Ellipsis]\\\", ButtonStyle->\\\"RefGuideLinkText\\\", ButtonFrame->None, \
> ButtonData:>\\\"General::partw\\\"]\\)\"\>"}]\)
>  From In[31]:=
> \!\(\*
>   RowBox[{\(Part::"partw"\),
>     ":", "\<\"Part \\!\\(106\\) of \\!\\({40.37`, 49.31`, 60.41`, 68.13`, \
> 70.44`, 64.32`, 64.6`, 47.77`, 70.39`, 95, \\(\\(\[LeftSkeleton] 95 \
> \[RightSkeleton]\\)\\)}\\) does not exist. \\!\\(\\*ButtonBox[\\\"More\
> \[Ellipsis]\\\", ButtonStyle->\\\"RefGuideLinkText\\\", ButtonFrame->None, \
> ButtonData:>\\\"General::partw\\\"]\\)\"\>"}]\)
>  From In[31]:=
> \!\(\*
>   RowBox[{\(Part::"partw"\),
>     ":", "\<\"Part \\!\\(107\\) of \\!\\({40.37`, 49.31`, 60.41`, 68.13`, \
> 70.44`, 64.32`, 64.6`, 47.77`, 70.39`, 95, \\(\\(\[LeftSkeleton] 95 \
> \[RightSkeleton]\\)\\)}\\) does not exist. \\!\\(\\*ButtonBox[\\\"More\
> \[Ellipsis]\\\", ButtonStyle->\\\"RefGuideLinkText\\\", ButtonFrame->None, \
> ButtonData:>\\\"General::partw\\\"]\\)\"\>"}]\)
>  From In[31]:=
> \!\(\*
>   RowBox[{\(General::"stop"\),
>     ":", "\<\"Further output of \\!\\(Part :: \\\"partw\\\"\\) will be \
> suppressed during this calculation. \
> \\!\\(\\*ButtonBox[\\\"More\[Ellipsis]\\\",
> ButtonStyle->\\\"RefGuideLinkText\
> \\\", ButtonFrame->None, ButtonData:>\\\"General::stop\\\"]\\)\"\>"}]\)
>  From In[31]:=
>
> No more memory available.
> Mathematica kernel has shut down.
> Try quitting other applications and then retry.
>  From In[5]:=
> Part::"partw":"Part \!\(106\) of \!\({40.37`, 49.31`, 60.41`, 68.13`,
> 70.44`, \
> 64.32`, 64.6`, 47.77`, 70.39`, 95, \(\(\[LeftSkeleton] 95 \
> \[RightSkeleton]\)\)}\) does not exist.
> \!\(\*ButtonBox[\"More\[Ellipsis]\", \
> ButtonStyle->\"RefGuideLinkText\", ButtonFrame->None, \
> ButtonData:>\"General::partw\"]\)"
>  From In[5]:=
> Part::"partw":"Part \!\(106\) of \!\({40.37`, 49.31`, 60.41`, 68.13`,
> 70.44`, \
> 64.32`, 64.6`, 47.77`, 70.39`, 95, \(\(\[LeftSkeleton] 95 \
> \[RightSkeleton]\)\)}\) does not exist.
> \!\(\*ButtonBox[\"More\[Ellipsis]\", \
> ButtonStyle->\"RefGuideLinkText\", ButtonFrame->None, \
> ButtonData:>\"General::partw\"]\)"
>  From In[5]:=
> Part::"partw":"Part \!\(107\) of \!\({40.37`, 49.31`, 60.41`, 68.13`,
> 70.44`, \
> 64.32`, 64.6`, 47.77`, 70.39`, 95, \(\(\[LeftSkeleton] 95 \
> \[RightSkeleton]\)\)}\) does not exist.
> \!\(\*ButtonBox[\"More\[Ellipsis]\", \
> ButtonStyle->\"RefGuideLinkText\", ButtonFrame->None, \
> ButtonData:>\"General::partw\"]\)"
>  From In[5]:=
> General::"stop":"Further output of \!\(Part :: \"partw\"\) will be
> suppressed \
> during this calculation. \!\(\*ButtonBox[\"More\[Ellipsis]\", \
> ButtonStyle->\"RefGuideLinkText\", ButtonFrame->None, \
> ButtonData:>\"General::stop\"]\)"
>
> Which, if the above doesn't come out (using DrBob's "
> Copy As InputForm"), says I am looking at non-existent parts of the
> matrix (106, etc) and also running out of memory on this calculation.
>
> Thanks for any suggestions.
>
> --Adam Getchell
>
>
>
>



-- 
DrBob at bigfoot.com


  • References:
  • Prev by Date: Re: SQLSelect
  • Next by Date: NDSolve, InterpolatingFunction objects and Splines
  • Previous by thread: Table function
  • Next by thread: RationalApproximation