Re: is there a better way to iterate this?

*To*: mathgroup at smc.vnet.net*Subject*: [mg77858] Re: [mg77820] is there a better way to iterate this?*From*: "dan siegal-gaskins" <dantimatter at gmail.com>*Date*: Mon, 18 Jun 2007 07:04:02 -0400 (EDT)*References*: <17667533.1182085910140.JavaMail.root@m35>

Hi DrMajorBob, > Table[N[Correlation[Flatten[foo[[i]], 1][[All, 1]], > Flatten[foo[[i]], 1][[All, 2]], {i, Length[foo]}]; > > has illegal syntax. There's a missing bracket somewhere. sorry about that. that table should be: bar=Table[ N[Correlation[Flatten[foo[[i]],1][[All,1]], Flatten[foo[[i]],1][[All,2]]]],{i, Length[foo]}]; these tables are embedded in a much larger notebook, and for simplicity i took out a couple of lines where i use DeleteCases[] and some other housekeeping functions. i also renamed the variables foo, bar, etc to make the question clearer. for these reasons, i think posting the actual notebook would probably do this thread more harm than good. :) all that's really important is that listA and listB are lists of 21 sublists, and each sublist has 18 elements. many thanks, dan > On Sun, 17 Jun 2007 05:13:05 -0500, dantimatter <dantimatter at gmail.com> > wrote: > > > > > Hello all, > > > > I have a question about a specific problem I'm working now. I would > > like to do 100000+ iterations of a series of operations, but the way > > I'm doing it now is slow and memory intensive. I know there's a > > better way to do it, but I just can't figure it out. The problem: I > > have two lists of 21 sublists: listA and listB, and I would like to > > shuffle the order of the sublists in listA around and then correlate > > them with listB. Here's how I'm doing it now: > > > > ------------------------ begin inefficient code > > ----------------------- > > > > randomizedlistA = Table[RandomPermutation[listA], {100000}]; > > > > listAlistBpairs = > > Table[Table[ > > Transpose[{listB[[j]], randomizedlistA[[k, j]]}, > > {j, Length[listB]}], {k, Length[randomizedlistA]}]; > > > > foo = > > Table[Table[ > > listAlistBpairs[[k, All, i]], {i, > > Length[listAlistBpairs[[k, 1]]]}], {k, > > Length[listAlistBpairs]}]; > > > > bar = > > Table[N[Correlation[Flatten[foo[[i]], 1][[All,1]], > > Flatten[foo[[i]], 1][[All, 2]], {i, > > Length[foo]}]; > > > > ------------------------ end inefficient code > > -------------------------- > > > > The variable 'bar' should now be a list of 100000 correlation > > coefficients. I realize this is probably bad Mathematica code, but I > > wrote it quickly and it worked great for smaller numbers of > > permutations. Can anyone suggest an easy way to do this better? > > > > Also, though it's not written in that code, I'd like to take those > > correlation coefficients and generate a probability distribution out > > of them. For 10000 permutations it comes out as a beautiful > > Gaussian; I can only assume it's going to look even better for > > 100000+ permutations. Any suggestions on how to generate a > > probability distribution, and how to calculate P-values from that > > distribution, would also be extremely helpful. > > > > Many thanks, > > Dan > > > > > > > > > > -- > DrMajorBob at bigfoot.com >