Re: Re: Out of memory.

*To*: mathgroup at smc.vnet.net*Subject*: [mg89259] Re: [mg89251] Re: Out of memory.*From*: Richard Bowles <watsonhb at arczip.com>*Date*: Mon, 2 Jun 2008 04:30:22 -0400 (EDT)*References*: <g1tjji$kh9$1@smc.vnet.net> <200806012033.QAA02073@smc.vnet.net>

Thanks Daniel and David for taking the time to reply. I have added a more complete sample of the code, with a simplified version of the matrix (10x10), that takes the 4th power of the matrix. This runs in a few mins. It will produce a couple of 1/0 division errors, but these disappear with increasing matrix size. A couple of additional points might also be of help. 1. The matrix is algebraic, so I was expecting memory problems with the MatrixPower operation, but this is actually completed for the full problem. The out of memory occurs once it has entered the loop for extracting the coefficients and exponents. The error occurs after a number of cycles has been completed. I plotted out MemoryInUse[] as a function of the number of inner loop cycles and see a linear increase in the memory consumed with one large jump in memory use part way through. This suggest me that it might be possible to release the memory consumed at each step but I have been unable to figure out how. However, there is an Expand[] operation in the loop over all the elements of the matrix, and this polynomial becomes extremely large with increasing power and it might be this point that ultimately does me in rather than the incremental memory consumption of the inner loop. 2. The configuration of the machine is correct, but I dont know how much of this memory Mathematica has access to. cheers Richard Here is the code. m1 = {{l^d1*n^u, 0, l^d2*n^u, l^d1*n^u, 0, l^d3*n^v, 0, 0, 0, l^d7*n^u2}, {l^d3*n^v, 0, 0, 0, l^d3*n^v, 0, l^d2*n^u, 0, 0, l^d5*n^v}, {0, 0, 0, 0, 0, 0, l^d4*n^v, 0, 0 , 0}, {0, 0, 0, 0, 0, 0, 0, l^d4*n^v, 0, 0}, {0, l^d3*n^v, l^d5*n^w, l^d1*n^u, l^d1*n^u, 0, 0, 0, l^d7*n^u2 , 0}, {0, l^d2*n^u, l^d3*n^v, 0, l^d3*n^v, 0, 0, 0, l^d5*n^v, 0}, {l^d5*n^w, 0, 0, l^d6*n^v, l^d1*n^u, l^d3*n^v, 0, 0, 0, 0}, {l^d2*n^u, l^d3*n^v, l^d6*n^v, 0, 0, 0, 0, 0, 0, 0}, {l^d7*n^u2, 0, 0, 0, 0, l^d5*n^v, 0, 0, 0, 0}, {0, l^d5*n^v, 0, 0, l^d7*n^u2, 0, 0, 0, 0, 0}} d1 = 1.1111111111111 d2 = 2.31229816711035 d3 = 3.81224989991992 d4 = 2.95769816711035 d5 = 1.27884902222222 d6 = 1.64544826719043 d7 = 2.47884908355518 d8 = 1.29109898347510 u = 2.50 v = 3.000 w = 3.500000 u2 = 4.000000 m2 = MatrixPower[m1, 4] s = OpenWrite["denld.dat"]; s1 = OpenWrite["coefld.dat"]; s2 = OpenWrite["update.dat"]; s = OpenWrite["a.dat"]; s1 = OpenWrite["b.dat"]; s2 = OpenWrite["update.dat"]; For[i = 0, i < 10, i++; For[j = 0, j < 10, j++; eepol = Expand[m2[[i, j]]]; k = Length[eepol]; Print[i, j]; Print[k] For[i1 = 0, i1 < k, i1++; w0[l_, n_] = Part[eepol, i1]; x1 = Exponent[w0[l, n], l]; x2 = Exponent[w0[l, n], n]; x3 = Coefficient[ w0[l, n], l^x1*n^x2]; dn = x2*0.25*Pi/(1.95*x1); Write[s1, x3]; Write[s, dn]; Write[s2, "dn=", dn, "i=", i, "j=", j]; ] ]] Close["denld.dat"]; Close["coefld.dat"]; Close["update.dat"]; On 1-Jun-08, at 2:33 PM, David Bailey wrote: > Richard Bowles wrote: >> I am rather new to mathematica so it is likely that I am doing some >> rather naive things. >> >> I have m1X20 matrix with matrix elements m1_ij=Cij P^lij S^nij. >> >> nij and lij are real numbers. Cij =1 or 0. >> >> I take the power matrix of m1^N >> m2 = MatrixPower[m1, N] >> >> Then need o extract the exponents of P and S and the coefficients of >> each of the terms in the resulting polynomial and write them to file. >> I do the following: >> >> For[i = 0, i < 20, i++; >> For[j = 0, j < 20, j++; >> >> eepol = Expand[m2[[i, j]]]; >> k = Length[eepol]; >> >> >> >> For[i1 = 0, i1 < k, i1++; >> >> w0[l_, n_] = Part[eepol, i1]; >> x1 = Exponent[w0[l, n], l]; >> x2 = Exponent[w0[l, n], n]; >> x3 = Coefficient[ w0[l, n], l^x1*n^x2]; >> dn = x2*0.25*Pi/(1.95*x1); >> >> Write[s1, x3]; >> >> Write[s, dn]; >> ] >> >> ]] >> >> This works slowly, but fine when N=10 but I run out of memory on a >> 128GB machine when N. I plotted the memory use as a function of >> cycles through the inner loop and find that it grows linearly which >> might account for the out of memory since there are a large number of >> terms. >> >> If anyone can suggest a faster and more memory efficient method it >> would be much appreciated. >> >> Cheers >> Richard >> > It is not totally clear to me if your matrix contains terms which are > algebraic expressions, or just numbers. I think you are talking > about a > matrix containing algebraic expressions, in which case the size of > those > expressions will rise rather fast as you do MatrixPower, so the memory > limit may be genuine. Symbolic expressions soak up a lot of memory and > CPU power as compared with numerical values, and they typically double > in size each time they are combined in some way (added or > multiplied, say). > > My first suggestion would be to post a complete example (as simple as > possible) so that we can play with it. > > Secondly it is extremely important to ensure that Mathematica can > actually use all your 128 GB (a very large figure - are you sure you > are > right?) Assuming you are using Windows, make sure you have the 64-bit > version of the OS loaded. Also, if Mathematica were installed before > the > change of OS, I would remove it and re-install (I am not certain if > this > step is necessary, but my guess is that it would be). > > David Bailey > http://www.dbaileyconsultancy.co.uk > >

**References**:**Re: Out of memory.***From:*David Bailey <dave@Remove_Thisdbailey.co.uk>

**SetOptions[TableForm] doesn't seem to work?**

**Re: Default location for Exported files?**

**Re: Out of memory.**

**Re: Out of memory.**