MathGroup Archive 2004

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: shuffling 10^8 numbers

  • To: mathgroup at
  • Subject: [mg53204] Re: [mg53180] shuffling 10^8 numbers
  • From: János <janos.lobb at>
  • Date: Tue, 28 Dec 2004 23:12:38 -0500 (EST)
  • References: <>
  • Sender: owner-wri-mathgroup at

First, try to set $HistoryLength to a needed finite number. That will 
free memory of Ins and Outs.
Second, try to write back to file intermediate results and quit the 
kernel. That is a pain, but... On some OSs it does not work - like OSX 
- and even quitting Mathematica does not help to give back the memory. 
Rebooting helps all the time.
Third, get a 64bit OS and 64bit Mathematica on it.

On Dec 28, 2004, at 6:30 AM, George Szpiro wrote:

> Hi,
> I am trying to shuffle 10^8 numbers stored in the file GG.doc in the 
> root directory. (Size of GG.doc appros 360 MB)
> Accorrding to previous suggestions from this group I try to shuffle 
> them witht he following program:
> GG=OpenRead["c:\GG.doc"];
> AA=ReadList[GG];
>  Timing[
>  OrigList=Table[AA];
>  p=RandomPermutation@Length@OrigList;
>  ShuffledList=OrigList[[p]];
> But the file is far too big. I can read it but then I get the 
> following error message:
> <<No more memory available. Mathematica kernel has shut down. Try 
> quitting other applications and then retry.>>
> No other programs are open, so I guess I am at the limit. Can anybody 
> suggest a workaround? Is there a possibility to shuffle numbers 
> without loading them all into memory simultaneously?
> NEW IDEA: I thought there might be a possibility of just reading one 
> single number each time from the file GG.doc, and putting them into a 
> randomly chosen slot in a new file.
> Any answeres greatly appreciated to:
> george at
> Thanks,
> George
"The shortest route between two points is the middleman" Ayn Rand

  • Prev by Date: Re: N[] does not work inside Replace[] ?
  • Next by Date: Re: Re: Condition , FullSimplify
  • Previous by thread: Re: shuffling 10^8 numbers
  • Next by thread: Re: shuffling 10^8 numbers