Re: Re: Out of memory.

• To: mathgroup at smc.vnet.net
• Subject: [mg89259] Re: [mg89251] Re: Out of memory.
• From: Richard Bowles <watsonhb at arczip.com>
• Date: Mon, 2 Jun 2008 04:30:22 -0400 (EDT)
• References: <g1tjji\$kh9\$1@smc.vnet.net> <200806012033.QAA02073@smc.vnet.net>

```Thanks Daniel and David for taking the time to reply.

I have added a more complete sample of the code, with a simplified
version of the matrix (10x10),  that takes the 4th power of the
matrix. This runs in a few mins. It will produce a couple of 1/0
division errors, but these disappear with increasing matrix size.

A couple of additional points might also be of help.

1. The matrix is algebraic, so I was expecting memory problems with
the MatrixPower operation, but this is actually completed for the full
problem. The out of memory occurs once it has entered the loop for
extracting the coefficients and exponents. The error occurs after a
number of cycles has been completed. I plotted out MemoryInUse[] as a
function of the number of inner loop cycles and see a linear increase
in the memory consumed with one large jump in memory use part way
through. This suggest me that it might be possible to release the
memory consumed at each step but I have been unable to figure out how.
However, there is an Expand[] operation in the loop over all the
elements of the matrix, and this polynomial becomes extremely large
with increasing power and it might be this point that ultimately does
me in rather than the incremental memory consumption of the inner loop.

2. The configuration of the machine is correct, but I dont know how

cheers
Richard

Here is the code.

m1 = {{l^d1*n^u, 0, l^d2*n^u, l^d1*n^u, 0, l^d3*n^v, 0, 0,  0,
l^d7*n^u2}, {l^d3*n^v, 0, 0, 0, l^d3*n^v, 0, l^d2*n^u, 0, 0,
l^d5*n^v}, {0, 0, 0, 0, 0, 0, l^d4*n^v, 0, 0 , 0}, {0, 0, 0, 0,
0,
0, 0, l^d4*n^v, 0, 0}, {0, l^d3*n^v, l^d5*n^w, l^d1*n^u,
l^d1*n^u,
0, 0, 0, l^d7*n^u2 , 0},
{0, l^d2*n^u, l^d3*n^v, 0, l^d3*n^v, 0, 0, 0, l^d5*n^v,
0}, {l^d5*n^w, 0, 0, l^d6*n^v, l^d1*n^u, l^d3*n^v, 0, 0, 0,
0}, {l^d2*n^u, l^d3*n^v, l^d6*n^v, 0, 0, 0, 0, 0, 0,
0},
{l^d7*n^u2, 0, 0, 0, 0, l^d5*n^v, 0, 0, 0, 0}, {0, l^d5*n^v,
0, 0, l^d7*n^u2, 0, 0, 0, 0, 0}}

d1 = 1.1111111111111
d2 = 2.31229816711035
d3 = 3.81224989991992
d4 = 2.95769816711035
d5 = 1.27884902222222
d6 = 1.64544826719043
d7 = 2.47884908355518
d8 = 1.29109898347510

u = 2.50
v = 3.000
w = 3.500000
u2 = 4.000000

m2 = MatrixPower[m1, 4]

s = OpenWrite["denld.dat"];
s1 = OpenWrite["coefld.dat"];
s2 = OpenWrite["update.dat"];

s = OpenWrite["a.dat"];
s1 = OpenWrite["b.dat"];
s2 = OpenWrite["update.dat"];

For[i = 0, i < 10, i++;
For[j = 0, j < 10, j++;

eepol = Expand[m2[[i, j]]];
k = Length[eepol];
Print[i, j];
Print[k]

For[i1 = 0, i1 < k, i1++;

w0[l_, n_] = Part[eepol, i1];
x1 = Exponent[w0[l, n], l];
x2 = Exponent[w0[l, n], n];
x3 = Coefficient[ w0[l, n], l^x1*n^x2];
dn = x2*0.25*Pi/(1.95*x1);
Write[s1, x3];

Write[s, dn];
Write[s2, "dn=", dn, "i=", i, "j=", j];

]

]]

Close["denld.dat"];
Close["coefld.dat"];
Close["update.dat"];

On 1-Jun-08, at 2:33 PM, David Bailey wrote:

> Richard Bowles wrote:
>> I am rather new to mathematica so it is likely that I am doing some
>> rather naive things.
>>
>> I have m1X20 matrix with matrix elements m1_ij=Cij P^lij S^nij.
>>
>> nij and lij are real numbers. Cij =1 or 0.
>>
>> I take the power matrix of m1^N
>> m2 = MatrixPower[m1, N]
>>
>> Then need o extract the exponents of P and S and the coefficients of
>> each of the terms in the resulting polynomial and write them to file.
>> I do the following:
>>
>> For[i = 0, i < 20, i++;
>>   For[j = 0, j < 20, j++;
>>
>>     eepol = Expand[m2[[i, j]]];
>>     k = Length[eepol];
>>
>>
>>
>>      For[i1 = 0, i1 < k, i1++;
>>
>>        w0[l_, n_] = Part[eepol, i1];
>>        x1 = Exponent[w0[l, n], l];
>>          x2 = Exponent[w0[l, n], n];
>>        x3 = Coefficient[ w0[l, n], l^x1*n^x2];
>>        dn = x2*0.25*Pi/(1.95*x1);
>>
>>         Write[s1, x3];
>>
>>        Write[s, dn];
>>         ]
>>
>>     ]]
>>
>> This works slowly, but fine when N=10 but I run out of memory on a
>> 128GB machine when N. I plotted the memory use as a function of
>> cycles through the inner loop and find that it grows linearly which
>> might account for the out of memory since there are a large number of
>> terms.
>>
>> If anyone can suggest a faster and more memory efficient method it
>> would be much appreciated.
>>
>> Cheers
>> Richard
>>
> It is not totally clear to me if your matrix contains terms which are
> algebraic expressions, or just numbers. I think you are talking
> matrix containing algebraic expressions, in which case the size of
> those
> expressions will rise rather fast as you do MatrixPower, so the memory
> limit may be genuine. Symbolic expressions soak up a lot of memory and
> CPU power as compared with numerical values, and they typically double
> in size each time they are combined in some way (added or
> multiplied, say).
>
> My first suggestion would be to post a complete example (as simple as
> possible) so that we can play with it.
>
> Secondly it is extremely important to ensure that Mathematica can
> actually use all your 128 GB (a very large figure - are you sure you
> are
> right?) Assuming you are using Windows, make sure you have the 64-bit
> version of the OS loaded. Also, if Mathematica were installed before
> the
> change of OS, I would remove it and re-install (I am not certain if
> this
> step is necessary, but my guess is that it would be).
>
> David Bailey
> http://www.dbaileyconsultancy.co.uk
>
>

```

• Prev by Date: SetOptions[TableForm] doesn't seem to work?
• Next by Date: Re: Default location for Exported files?
• Previous by thread: Re: Out of memory.
• Next by thread: Re: Out of memory.