Re: shuffling 10^8 numbers
- To: mathgroup at smc.vnet.net
- Subject: [mg53208] Re: [mg53180] shuffling 10^8 numbers
- From: yehuda ben-shimol <benshimo at bgu.ac.il>
- Date: Tue, 28 Dec 2004 23:12:49 -0500 (EST)
- References: <200412281130.GAA26970@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
I tried your code under Linux SUSE 9.1 with Mathematica 5.0.0. with 512MB (a modest laptop). I generated a=Range[10^8]; The table was generated. p=RandomPermutation@Length@a Still working (took few minutes) The crash happened with b=a[[p]]; So another way needed to be tried only at that point. Instead of inserting all the indexes at once, do it one by one with Map as follows b=a[[#]]&/@a; I checked it and it is working with my laptop (although it took a while due to swapping) yehuda George Szpiro wrote: >Hi, > >I am trying to shuffle 10^8 numbers stored in the file GG.doc in the root >directory. (Size of GG.doc appros 360 MB) > >Accorrding to previous suggestions from this group I try to shuffle them >witht he following program: > >GG=OpenRead["c:\GG.doc"]; >AA=ReadList[GG]; > Timing[ > OrigList=Table[AA]; > p=RandomPermutation@Length@OrigList; > ShuffledList=OrigList[[p]]; > > >But the file is far too big. I can read it but then I get the following >error message: > ><<No more memory available. Mathematica kernel has shut down. Try quitting >other applications and then retry.>> > >No other programs are open, so I guess I am at the limit. Can anybody >suggest a workaround? Is there a possibility to shuffle numbers without >loading them all into memory simultaneously? > >NEW IDEA: I thought there might be a possibility of just reading one single >number each time from the file GG.doc, and putting them into a randomly >chosen slot in a new file. > >Any answeres greatly appreciated to: >george at netvision.net.il > >Thanks, >George > > > >
- References:
- shuffling 10^8 numbers
- From: George Szpiro <george@netvision.net.il>
- shuffling 10^8 numbers