Re: Shuffling 10^8 numbers: memory problems

• To: mathgroup at smc.vnet.net
• Subject: [mg53302] Re: [mg53295] Shuffling 10^8 numbers: memory problems
• From: János <janos.lobb at yale.edu>
• Date: Wed, 5 Jan 2005 01:21:17 -0500 (EST)
• References: <200501040814.DAA27253@smc.vnet.net>
• Sender: owner-wri-mathgroup at wolfram.com

```On Jan 4, 2005, at 3:14 AM, George Szpiro wrote:

> The problem of quadratic time seems to have been solved by using
>
> aa = Developer`ToPackedArray[aa];
>
> after reading in the data.
>
> But now the Mathematica kernel hits the limit again at 2*10^7 data: it
> shuts down
> MemoryInUse[] shows about 82 MB were used, but my machine has 704MB
> RAM installed! What is happening?
>
> Is there a way to work around that one? I need to shuffle five times as
> many integer numbers, i.e., I would need about 420 MB memory.
>
> Would it maybe help to start my machine in Safe Mode, so that no other
> processes run, and thus get the use of nearly the full 704 MB memory?
I guess, you are using some kind of Windows here. Try to increase the
amount of virtual memory in the machine with a few GigaByte.

János

> (By the way, the 10^8 numbers are integers that are not larger than
>
> Thanks,
>  George
>
> The program I used:
>
> Clear[SSS,aa];
> SSS=20000000;  (*I need to use 100000000*)
>
> aa = Developer`ToPackedArray[aa];
>
> SetAttributes[shuffleSet, HoldFirst];
> shuffleSet[LL_] := Module[{n=SSS, rand},
>  Do[
>  rand = Random[Integer, {j,n}];
>  LL[[{j,rand}]] = LL[[{rand,j}]],
>  {j,n}];
>  ]
>
> Timing[shuffleSet[aa]]
>
> <<Utilities`CleanSlate`
>
>  OpenWrite["c:ShuffledGG.doc"];
> Put[OutputForm[aa],"c:ShuffledGG.doc"]
> Close["c:ShuffledGG.doc"];
>
>
>
--------
"At a risk of being called sexist, ageist and French, if you put
multimedia, a leather skirt and lipstick on a grandmother and take her
to a night club, she's still not going to get lucky." -- Jean Louis
Gassee (of Be) on Windows 2000 being "multimedia."

```

• Prev by Date: Converting between Spherical and Cartesian coordinates
• Next by Date: Re: Shuffling 10^8 numbers: memory problems
• Previous by thread: Shuffling 10^8 numbers: memory problems
• Next by thread: Re: Shuffling 10^8 numbers: memory problems