Re: is there a better way to iterate this?

*To*: mathgroup at smc.vnet.net*Subject*: [mg77865] Re: is there a better way to iterate this?*From*: Ray Koopman <koopman at sfu.ca>*Date*: Mon, 18 Jun 2007 07:07:37 -0400 (EDT)*References*: <f532ln$3ab$1@smc.vnet.net>

On Jun 17, 3:33 am, dantimatter <dantimat... at gmail.com> wrote: > Hello all, > > I have a question about a specific problem I'm working now. I would > like to do 100000+ iterations of a series of operations, but the way > I'm doing it now is slow and memory intensive. I know there's a > better way to do it, but I just can't figure it out. The problem: I > have two lists of 21 sublists: listA and listB, and I would like to > shuffle the order of the sublists in listA around and then correlate > them with listB. Here's how I'm doing it now: > > --------------------- begin inefficient code --------------------- > > randomizedlistA = Table[RandomPermutation[listA], {100000}]; > > listAlistBpairs = Table[Table[ > Transpose[{listB[[j]],randomizedlistA[[k,j]]}, > {j, Length[listB]}], {k, Length[randomizedlistA]}]; > > foo = Table[Table[ > listAlistBpairs[[k, All, i]], {i, > Length[listAlistBpairs[[k, 1]]]}], {k, > Length[listAlistBpairs]}]; > > bar = > Table[N[Correlation[Flatten[foo[[i]], 1][[All,1]], > Flatten[foo[[i]], 1][[All, 2]], {i, > Length[foo]}]; > > ---------------------- end inefficient code ---------------------- > > The variable 'bar' should now be a list of 100000 correlation > coefficients. I realize this is probably bad Mathematica code, but > I wrote it quickly and it worked great for smaller numbers of > permutations. Can anyone suggest an easy way to do this better? > > Also, though it's not written in that code, I'd like to take those > correlation coefficients and generate a probability distribution > out of them. For 10000 permutations it comes out as a beautiful > Gaussian; I can only assume it's going to look even better for > 100000+ permutations. Any suggestions on how to generate a > probability distribution, and how to calculate P-values from that > distribution, would also be extremely helpful. > > Many thanks, > Dan a = #/Norm@Flatten@#&[listA - Mean@Flatten@listA]; b = #/Norm@#&[Flatten@listB - Mean@Flatten@listB]; bar = Table[Flatten[a[[Ordering@Table[Random[],{21}]]]],{1*^5}].b