Re: Large SparseArray in Mathematica 9 and the predictive interface

*To*: mathgroup at smc.vnet.net*Subject*: [mg129306] Re: Large SparseArray in Mathematica 9 and the predictive interface*From*: Bill Rowe <readnews at sbcglobal.net>*Date*: Mon, 31 Dec 2012 19:44:51 -0500 (EST)*Delivered-to*: l-mathgroup@mail-archive0.wolfram.com*Delivered-to*: l-mathgroup@wolfram.com*Delivered-to*: mathgroup-newout@smc.vnet.net*Delivered-to*: mathgroup-newsend@smc.vnet.net

On 12/30/12 at 8:50 PM, jmmv73 at gmail.com wrote: >I was generating a banded sparse array with the following code in >the new Mathematica 9.0: >gmat[K_, l_, m_] := >Module[{bandU, bandL, bandD, res}, >bandU = Table[l, {K - 1}]; bandL = Table[m, {K - 1}]; bandD = Table[-l - m, {K}]; bandD[[1]] = -l; bandD[[K]] = -m; >res = DiagonalMatrix[SparseArray[bandU], 1] + >DiagonalMatrix[SparseArray[bandD]] + >DiagonalMatrix[SparseArray[bandL], -1]; >Return[res]; >]; >when I typed (without semicolon): >gmat[10000, 0.8, 1] >...everything went not so good, a 10000x10000 sparse matrix was >created (what was ok), but the kernel went nuts with the memory use, >some gigabytes for a small sparsearray (You can try with greater >parameters, but starting with K=5000 things go weird) >I think the problem has to do with the "predictive interface" that >tries to generate the normal form of the matrix, so it uses a lot of >memory (and even produces a kernel crash). A few comments: First, I cannot reproduce your results. That using your code slightly cleaned up, I don't see any problems. By slightly cleaned up I mean replacing K (which as a built-in meaning) with k and eliminating the local variable res which isn't needed to get: gmat[k_, l_, m_] := Module[{bandU, bandL, bandD}, bandU = Table[l, {k - 1}]; bandL = Table[m, {k - 1}]; bandD = Table[-l - m, {k}]; bandD[[1]] = -l; bandD[[k]] = -m; DiagonalMatrix[SparseArray[bandU], 1] + DiagonalMatrix[SparseArray[bandD]] + DiagonalMatrix[SparseArray[bandL], -1]] And this code can be further simplified to: gMat[k_, l_, m_] := SparseArray[{Band[{1, 1}] -> -m - l, Band[{1, 2}] -> l, Band[{2, 1}] -> m}, {k, k}] + SparseArray[{{1, 1} -> m, {k, k} -> l}] But I do see a difference between version 8 and 9 with respect to the amount of memory used. That is: In[1]:= gMat[k_, l_, m_] := SparseArray[{Band[{1, 1}] -> -m - l, Band[{1, 2}] -> l, Band[{2, 1}] -> m}, {k, k}] + SparseArray[{{1, 1} -> m, {k, k} -> l}] In[2]:= a = MemoryInUse[]; gMat[10000, .8, 1]; MemoryInUse[] - a Out[3]= 1003920 In[4]:= $Version Out[4]= 9.0 for Mac OS X x86 (64-bit) (November 20, 2012) and In[1]:= gMat[k_, l_, m_] := SparseArray[{Band[{1, 1}] -> -m - l, Band[{1, 2}] -> l, Band[{2, 1}] -> m}, {k, k}] + SparseArray[{{1, 1} -> m, {k, k} -> l}] In[2]:= a = MemoryInUse[]; gMat[10000, .8, 1]; MemoryInUse[] - a Out[3]= 493024 In[4]:= $Version Out[4]= 8.0 for Mac OS X x86 (64-bit) (October 5, 2011) In version 9 on my system, the array requires a bit more than twice the memory that version 8 requires. So, there may be something to your speculation about the predictive interface. One other thing. I should point out I have 16GB of RAM installed which may be why I am not seeing problems with your code.