MathGroup Archive 2004

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: RandomReplacement

  • To: mathgroup at smc.vnet.net
  • Subject: [mg51669] Re: RandomReplacement
  • From: Andrzej Kozlowski <akoz at mimuw.edu.pl>
  • Date: Wed, 27 Oct 2004 23:44:58 -0400 (EDT)
  • References: <20041028000310.41708.qmail@web60508.mail.yahoo.com>
  • Sender: owner-wri-mathgroup at wolfram.com

On 28 Oct 2004, at 09:03, sean kim wrote:
>
> hi andrej and everyone
>
> I'm starting to think *maybe* the PackedArray isn't
> the problem. To reproduce the problem...

Wel, first you have to make clear the meaning of "problem". Packed 
Arrays are not a problem in themselves: they are a "good thing' for 
they speed things up greatly. The problem is that when Mathematica uses 
Packed Arrays it does things differently than when it does not; using a 
compiled code that is impossible or at least very difficult for the 
user to modify. In our case the problem was that the packed array way 
of making a table of random reals uses the built in Random[] generator 
and is not affected by any changes the user makes to the global 
Random[] function. So in my RandomReplacement package I used a trick to 
temporarily suppress using packed arrays. The effect is to create a big 
slow down. But also note the word "temporarily"!


>
> In[1]:= SeedRandom[5];
> Timing[First[real1=Table[Random[],{249}]]]
> Developer`PackedArrayQ[real1]
>
> Out[2]= {0. Second,0.786599}
> Out[3]= False
>
> In[4]:= SeedRandom[5];
> Timing[First[real2=Table[Random[],{250}]]]
> Developer`PackedArrayQ[real2]
>
> Out[5]= {0. Second,0.786599}
> Out[6]= True
>
> and there it is.

Yes, this illustrates the way Packed Arrays work here. You only get a 
packed array if you use the function Random[] in table and the table is 
large enough.
> now...
>
> Quit
>
> In[1]:= Needs["Statistics`ContinuousDistributions`"];
> In[2]:= uni= UniformDistribution[0, 1]
> Out[2]= UniformDistribution[0,1]
>
> In[3]:= SeedRandom[5];
> Timing[First[uni1=Table[Random[uni],{249}]]]
> Developer`PackedArrayQ[uni1]
>
> Out[4]= {0. Second,0.786599}
> Out[5]= False
>
>
> In[6]:= SeedRandom[5];
> Timing[First[uni2=Table[Random[uni],{250}]]]
> Developer`PackedArrayQ[uni2]
>
> Out[7]= {0.01 Second,0.786599}
> Out[8]= False
>
> Without loading RandomReplacement, we see the
> packedarray problem is no longer present if we change
> way we use the Random.

Indded PackedArrays are not used if you make your random table in this 
way. What you ge tis "statistically" equivalent ot using Random[] but 
not equivalent to it in the way Mathemaitca does the computation. So 
this way could also be used as a "workaround". But notice the speed 
difference between this way of making a uniformly distributed table and 
the Random[] in way:


Needs["Statistics`ContinuousDistributions`"];


SeedRandom[5];


Timing[First[Table[Random[UniformDistribution[]],{10^6}]]]

{13.55 Second,0.786599}


SeedRandom[5];


Timing[First[Table[Random[],{10^6}]]]

{0.63 Second,0.786599}


So although one can base a workaround on this and RandomReplacement it 
won't significantly improve the performance over what RandomReplacement 
does. When you do not use  random arrays the loss of performance is 
huge.

>
> Needs["Statistics`ContinuousDistributions`"];
> uni= UniformDistribution[0, 1]
> Random[uni]
>
> should be identical to Random[] ( according to help)
>
> but once we load the RandomReplacement,
>
> Quit
>
> In[1]:= <<RandomReplacement.m
> In[2]:= SeedRandom[5];
> Timing[First[real3=Table[Random[],{249}]]]
> Developer`PackedArrayQ[real3]
>
> Timing[First[real4=Table[Random[],{250}]]]
> Developer`PackedArrayQ[real4]
>
> Out[3]= {0. Second,0.66205}
> Out[4]= False
> Out[5]= {0. Second,0.793703}
> Out[6]= True
>
> problem with packedarray is still there.

No the "problem" is not there. I used Developer`ToPackedArray in my 
code to create a packed array as the final output, as would happen 
without the RandomReplacement package. So you you do get a packed array 
at the end, which should make things fast if you now perform certain 
computations on this output, but the PackedArray technology was 
suppressed during the making of the list: hence the performance hit.  
Unfortunately i have not found any way to significantly speed things 
over what RandomReplacement already does. Various tricks that I have 
tried produced an improvement of less than 10%, so in the end I did not 
bother trying to implement them. Basically any method of generating a 
random table that does not use the PackedArray technology will be much 
slower. But note that the PackedArray technology is not used if you 
generate random tables using any of the distributions defined in the 
Statistics` packages, so in such a case the loss of performance due to 
RandomReplacement` is much smaller.

Andrzej Kozlowski
Chiba, Japan
http://www.akikoz.net/~andrzej/
http://www.mimuw.edu.pl/~akoz/


  • Prev by Date: chiral Dirac six space
  • Next by Date: Converting Mathematica to MS Word
  • Previous by thread: Re: RandomReplacement
  • Next by thread: Integrate vs. NIntegrate