MathGroup Archive 2007

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: is there a better way to iterate this?

  • To: mathgroup at smc.vnet.net
  • Subject: [mg77865] Re: is there a better way to iterate this?
  • From: Ray Koopman <koopman at sfu.ca>
  • Date: Mon, 18 Jun 2007 07:07:37 -0400 (EDT)
  • References: <f532ln$3ab$1@smc.vnet.net>

On Jun 17, 3:33 am, dantimatter <dantimat... at gmail.com> wrote:
> Hello all,
>
> I have a question about a specific problem I'm working now.  I would
> like to do 100000+ iterations of a series of operations, but the way
> I'm doing it now is slow and memory intensive.  I know there's a
> better way to do it, but I just can't figure it out.  The problem: I
> have two lists of 21 sublists: listA and listB, and I would like to
> shuffle the order of the sublists in listA around and then correlate
> them with listB.  Here's how I'm doing it now:
>
> --------------------- begin inefficient code ---------------------
>
> randomizedlistA = Table[RandomPermutation[listA], {100000}];
>
> listAlistBpairs = Table[Table[
>                     Transpose[{listB[[j]],randomizedlistA[[k,j]]},
> {j, Length[listB]}], {k, Length[randomizedlistA]}];
>
> foo = Table[Table[
>             listAlistBpairs[[k, All, i]], {i,
>               Length[listAlistBpairs[[k, 1]]]}], {k,
>             Length[listAlistBpairs]}];
>
> bar =
>       Table[N[Correlation[Flatten[foo[[i]], 1][[All,1]],
> Flatten[foo[[i]], 1][[All, 2]], {i,
>           Length[foo]}];
>
> ---------------------- end inefficient code ----------------------
>
> The variable 'bar' should now be a list of 100000 correlation
> coefficients. I realize this is probably bad Mathematica code, but
> I wrote it quickly and it worked great for smaller numbers of
> permutations.  Can anyone suggest an easy way to do this better?
>
> Also, though it's not written in that code, I'd like to take those
> correlation coefficients and generate a probability distribution
> out of them.  For 10000 permutations it comes out as a beautiful
> Gaussian;  I can only assume it's going to look even better for
> 100000+ permutations.  Any suggestions on how to generate a
> probability distribution, and how to calculate P-values from that
> distribution, would also be extremely helpful.
>
> Many thanks,
> Dan

a = #/Norm@Flatten@#&[listA - Mean@Flatten@listA];
b = #/Norm@#&[Flatten@listB - Mean@Flatten@listB];
bar = Table[Flatten[a[[Ordering@Table[Random[],{21}]]]],{1*^5}].b



  • Prev by Date: Re: a beginner's question
  • Next by Date: Re: Trying to export two column table to text file
  • Previous by thread: is there a better way to iterate this?
  • Next by thread: Re: is there a better way to iterate this?