[Date Index] [Thread Index] [Author Index]

Re: shuffling 10^8 numbers

• To: mathgroup at smc.vnet.net
• Subject: [mg53208] Re: [mg53180] shuffling 10^8 numbers
• From: yehuda ben-shimol <benshimo at bgu.ac.il>
• Date: Tue, 28 Dec 2004 23:12:49 -0500 (EST)
• References: <200412281130.GAA26970@smc.vnet.net>
• Sender: owner-wri-mathgroup at wolfram.com

I tried your code  under Linux SUSE 9.1 with Mathematica 5.0.0. with 
512MB (a modest laptop).
I generated a=Range[10^8];
The table was generated.

p=RandomPermutation@Length@a
Still working (took few minutes)
The crash happened with 
b=a[[p]];

So another way needed to be tried only at that point.
Instead of inserting all the indexes at once, do it one by one with Map as follows
b=a[[#]]&/@a;
I checked it and it is working with my laptop (although it took a while due to swapping)
yehuda 




George Szpiro wrote:

>Hi,
>
>I am trying to shuffle 10^8 numbers stored in the file GG.doc in the root 
>directory. (Size of GG.doc appros 360 MB)
>
>Accorrding to previous suggestions from this group I try to shuffle them 
>witht he following program:
>
>GG=OpenRead["c:\GG.doc"];
>AA=ReadList[GG];
>  Timing[
>  OrigList=Table[AA];
>  p=RandomPermutation@Length@OrigList;
>  ShuffledList=OrigList[[p]];
>
>
>But the file is far too big. I can read it but then I get the following 
>error message:
>
><<No more memory available. Mathematica kernel has shut down. Try quitting 
>other applications and then retry.>>
>
>No other programs are open, so I guess I am at the limit. Can anybody 
>suggest a workaround? Is there a possibility to shuffle numbers without 
>loading them all into memory simultaneously?
>
>NEW IDEA: I thought there might be a possibility of just reading one single 
>number each time from the file GG.doc, and putting them into a randomly 
>chosen slot in a new file.
>
>Any answeres greatly appreciated to:
>george at netvision.net.il
>
>Thanks,
>George
>
>
>  
>

• References:
  • shuffling 10^8 numbers
    • From: George Szpiro <george@netvision.net.il>