Re: is there a better way to iterate this?
- To: mathgroup at smc.vnet.net
- Subject: [mg77865] Re: is there a better way to iterate this?
- From: Ray Koopman <koopman at sfu.ca>
- Date: Mon, 18 Jun 2007 07:07:37 -0400 (EDT)
- References: <f532ln$3ab$1@smc.vnet.net>
On Jun 17, 3:33 am, dantimatter <dantimat... at gmail.com> wrote:
> Hello all,
>
> I have a question about a specific problem I'm working now. I would
> like to do 100000+ iterations of a series of operations, but the way
> I'm doing it now is slow and memory intensive. I know there's a
> better way to do it, but I just can't figure it out. The problem: I
> have two lists of 21 sublists: listA and listB, and I would like to
> shuffle the order of the sublists in listA around and then correlate
> them with listB. Here's how I'm doing it now:
>
> --------------------- begin inefficient code ---------------------
>
> randomizedlistA = Table[RandomPermutation[listA], {100000}];
>
> listAlistBpairs = Table[Table[
> Transpose[{listB[[j]],randomizedlistA[[k,j]]},
> {j, Length[listB]}], {k, Length[randomizedlistA]}];
>
> foo = Table[Table[
> listAlistBpairs[[k, All, i]], {i,
> Length[listAlistBpairs[[k, 1]]]}], {k,
> Length[listAlistBpairs]}];
>
> bar =
> Table[N[Correlation[Flatten[foo[[i]], 1][[All,1]],
> Flatten[foo[[i]], 1][[All, 2]], {i,
> Length[foo]}];
>
> ---------------------- end inefficient code ----------------------
>
> The variable 'bar' should now be a list of 100000 correlation
> coefficients. I realize this is probably bad Mathematica code, but
> I wrote it quickly and it worked great for smaller numbers of
> permutations. Can anyone suggest an easy way to do this better?
>
> Also, though it's not written in that code, I'd like to take those
> correlation coefficients and generate a probability distribution
> out of them. For 10000 permutations it comes out as a beautiful
> Gaussian; I can only assume it's going to look even better for
> 100000+ permutations. Any suggestions on how to generate a
> probability distribution, and how to calculate P-values from that
> distribution, would also be extremely helpful.
>
> Many thanks,
> Dan
a = #/Norm@Flatten@#&[listA - Mean@Flatten@listA];
b = #/Norm@#&[Flatten@listB - Mean@Flatten@listB];
bar = Table[Flatten[a[[Ordering@Table[Random[],{21}]]]],{1*^5}].b