Services & ResourcesWolfram ForumsMathGroup Archive

[Date Index] [Thread Index] [Author Index]

Re: Re: Out of memory.

• To: mathgroup at smc.vnet.net
• Subject: [mg89259] Re: [mg89251] Re: Out of memory.
• From: Richard Bowles <watsonhb at arczip.com>
• Date: Mon, 2 Jun 2008 04:30:22 -0400 (EDT)
• References: <g1tjji$kh9$1@smc.vnet.net> <200806012033.QAA02073@smc.vnet.net>

Thanks Daniel and David for taking the time to reply.

I have added a more complete sample of the code, with a simplified  
version of the matrix (10x10),  that takes the 4th power of the  
matrix. This runs in a few mins. It will produce a couple of 1/0  
division errors, but these disappear with increasing matrix size.

A couple of additional points might also be of help.

1. The matrix is algebraic, so I was expecting memory problems with  
the MatrixPower operation, but this is actually completed for the full  
problem. The out of memory occurs once it has entered the loop for  
extracting the coefficients and exponents. The error occurs after a  
number of cycles has been completed. I plotted out MemoryInUse[] as a  
function of the number of inner loop cycles and see a linear increase  
in the memory consumed with one large jump in memory use part way  
through. This suggest me that it might be possible to release the  
memory consumed at each step but I have been unable to figure out how.  
However, there is an Expand[] operation in the loop over all the  
elements of the matrix, and this polynomial becomes extremely large  
with increasing power and it might be this point that ultimately does  
me in rather than the incremental memory consumption of the inner loop.

2. The configuration of the machine is correct, but I dont know how  
much of this memory Mathematica has access to.

cheers
Richard

Here is the code.

m1 = {{l^d1*n^u, 0, l^d2*n^u, l^d1*n^u, 0, l^d3*n^v, 0, 0,  0,
       l^d7*n^u2}, {l^d3*n^v, 0, 0, 0, l^d3*n^v, 0, l^d2*n^u, 0, 0,
       l^d5*n^v}, {0, 0, 0, 0, 0, 0, l^d4*n^v, 0, 0 , 0}, {0, 0, 0, 0,  
0,
       0, 0, l^d4*n^v, 0, 0}, {0, l^d3*n^v, l^d5*n^w, l^d1*n^u,  
l^d1*n^u,
       0, 0, 0, l^d7*n^u2 , 0},
     {0, l^d2*n^u, l^d3*n^v, 0, l^d3*n^v, 0, 0, 0, l^d5*n^v,
       0}, {l^d5*n^w, 0, 0, l^d6*n^v, l^d1*n^u, l^d3*n^v, 0, 0, 0,
       0}, {l^d2*n^u, l^d3*n^v, l^d6*n^v, 0, 0, 0, 0, 0, 0,
       0},
     {l^d7*n^u2, 0, 0, 0, 0, l^d5*n^v, 0, 0, 0, 0}, {0, l^d5*n^v,
       0, 0, l^d7*n^u2, 0, 0, 0, 0, 0}}

  d1 = 1.1111111111111
d2 = 2.31229816711035
d3 = 3.81224989991992
d4 = 2.95769816711035
d5 = 1.27884902222222
d6 = 1.64544826719043
d7 = 2.47884908355518
d8 = 1.29109898347510

u = 2.50
v = 3.000
w = 3.500000
u2 = 4.000000

m2 = MatrixPower[m1, 4]

s = OpenWrite["denld.dat"];
s1 = OpenWrite["coefld.dat"];
s2 = OpenWrite["update.dat"];

s = OpenWrite["a.dat"];
s1 = OpenWrite["b.dat"];
s2 = OpenWrite["update.dat"];

For[i = 0, i < 10, i++;
   For[j = 0, j < 10, j++;

     eepol = Expand[m2[[i, j]]];
     k = Length[eepol];
     Print[i, j];
   Print[k]



      For[i1 = 0, i1 < k, i1++;

        w0[l_, n_] = Part[eepol, i1];
        x1 = Exponent[w0[l, n], l];
          x2 = Exponent[w0[l, n], n];
        x3 = Coefficient[ w0[l, n], l^x1*n^x2];
        dn = x2*0.25*Pi/(1.95*x1);
         Write[s1, x3];

        Write[s, dn];
        Write[s2, "dn=", dn, "i=", i, "j=", j];

        ]

     ]]

Close["denld.dat"];
Close["coefld.dat"];
Close["update.dat"];

On 1-Jun-08, at 2:33 PM, David Bailey wrote:

> Richard Bowles wrote:
>> I am rather new to mathematica so it is likely that I am doing some
>> rather naive things.
>>
>> I have m1X20 matrix with matrix elements m1_ij=Cij P^lij S^nij.
>>
>> nij and lij are real numbers. Cij =1 or 0.
>>
>> I take the power matrix of m1^N
>> m2 = MatrixPower[m1, N]
>>
>> Then need o extract the exponents of P and S and the coefficients of
>> each of the terms in the resulting polynomial and write them to file.
>> I do the following:
>>
>> For[i = 0, i < 20, i++;
>>   For[j = 0, j < 20, j++;
>>
>>     eepol = Expand[m2[[i, j]]];
>>     k = Length[eepol];
>>
>>
>>
>>      For[i1 = 0, i1 < k, i1++;
>>
>>        w0[l_, n_] = Part[eepol, i1];
>>        x1 = Exponent[w0[l, n], l];
>>          x2 = Exponent[w0[l, n], n];
>>        x3 = Coefficient[ w0[l, n], l^x1*n^x2];
>>        dn = x2*0.25*Pi/(1.95*x1);
>>
>>         Write[s1, x3];
>>
>>        Write[s, dn];
>>         ]
>>
>>     ]]
>>
>> This works slowly, but fine when N=10 but I run out of memory on a
>> 128GB machine when N. I plotted the memory use as a function of
>> cycles through the inner loop and find that it grows linearly which
>> might account for the out of memory since there are a large number of
>> terms.
>>
>> If anyone can suggest a faster and more memory efficient method it
>> would be much appreciated.
>>
>> Cheers
>> Richard
>>
> It is not totally clear to me if your matrix contains terms which are
> algebraic expressions, or just numbers. I think you are talking  
> about a
> matrix containing algebraic expressions, in which case the size of  
> those
> expressions will rise rather fast as you do MatrixPower, so the memory
> limit may be genuine. Symbolic expressions soak up a lot of memory and
> CPU power as compared with numerical values, and they typically double
> in size each time they are combined in some way (added or  
> multiplied, say).
>
> My first suggestion would be to post a complete example (as simple as
> possible) so that we can play with it.
>
> Secondly it is extremely important to ensure that Mathematica can
> actually use all your 128 GB (a very large figure - are you sure you  
> are
> right?) Assuming you are using Windows, make sure you have the 64-bit
> version of the OS loaded. Also, if Mathematica were installed before  
> the
> change of OS, I would remove it and re-install (I am not certain if  
> this
> step is necessary, but my guess is that it would be).
>
> David Bailey
> http://www.dbaileyconsultancy.co.uk
>
>

• References:
  • Re: Out of memory.
    • From: David Bailey <dave@Remove_Thisdbailey.co.uk>