[Date Index] [Thread Index] [Author Index]

shuffling 10^8 numbers

• To: mathgroup at smc.vnet.net
• Subject: [mg53180] shuffling 10^8 numbers
• From: George Szpiro <george at netvision.net.il>
• Date: Tue, 28 Dec 2004 06:30:22 -0500 (EST)
• Sender: owner-wri-mathgroup at wolfram.com

Hi,

I am trying to shuffle 10^8 numbers stored in the file GG.doc in the root 
directory. (Size of GG.doc appros 360 MB)

Accorrding to previous suggestions from this group I try to shuffle them 
witht he following program:

GG=OpenRead["c:\GG.doc"];
AA=ReadList[GG];
  Timing[
  OrigList=Table[AA];
  p=RandomPermutation@Length@OrigList;
  ShuffledList=OrigList[[p]];


But the file is far too big. I can read it but then I get the following 
error message:

<<No more memory available. Mathematica kernel has shut down. Try quitting 
other applications and then retry.>>

No other programs are open, so I guess I am at the limit. Can anybody 
suggest a workaround? Is there a possibility to shuffle numbers without 
loading them all into memory simultaneously?

NEW IDEA: I thought there might be a possibility of just reading one single 
number each time from the file GG.doc, and putting them into a randomly 
chosen slot in a new file.

Any answeres greatly appreciated to:
george at netvision.net.il

Thanks,
George

• Follow-Ups:
  • Re: shuffling 10^8 numbers
    • From: Daniel Lichtblau <danl@wolfram.com>
  • Re: shuffling 10^8 numbers
    • From: Daniel Lichtblau <danl@wolfram.com>
  • Re: shuffling 10^8 numbers
    • From: DrBob <drbob@bigfoot.com>
  • Re: shuffling 10^8 numbers
    • From: János <janos.lobb@yale.edu>
  • Re: shuffling 10^8 numbers
    • From: yehuda ben-shimol <benshimo@bgu.ac.il>