Mathematica 9 is now available
Services & Resources / Wolfram Forums / MathGroup Archive
-----

MathGroup Archive 2012

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Large SparseArray in Mathematica 9 and the predictive interface

  • To: mathgroup at smc.vnet.net
  • Subject: [mg129306] Re: Large SparseArray in Mathematica 9 and the predictive interface
  • From: Bill Rowe <readnews at sbcglobal.net>
  • Date: Mon, 31 Dec 2012 19:44:51 -0500 (EST)
  • Delivered-to: l-mathgroup@mail-archive0.wolfram.com
  • Delivered-to: l-mathgroup@wolfram.com
  • Delivered-to: mathgroup-newout@smc.vnet.net
  • Delivered-to: mathgroup-newsend@smc.vnet.net

On 12/30/12 at 8:50 PM, jmmv73 at gmail.com wrote:

>I was generating a banded sparse array with the following code in
>the new Mathematica 9.0:

>gmat[K_, l_, m_] :=
>Module[{bandU, bandL, bandD, res},

>bandU = Table[l, {K - 1}]; bandL = Table[m, {K - 1}]; bandD = Table[-l - m, {K}]; bandD[[1]] = -l; bandD[[K]] = -m;

>res = DiagonalMatrix[SparseArray[bandU], 1] +
>DiagonalMatrix[SparseArray[bandD]] +
>DiagonalMatrix[SparseArray[bandL], -1];

>Return[res];
>];

>when I typed (without semicolon):

>gmat[10000, 0.8, 1]

>...everything went not so good, a 10000x10000 sparse matrix was
>created (what was ok), but the kernel went nuts with the memory use,
>some gigabytes for a small sparsearray (You can try with greater
>parameters, but starting with K=5000 things go weird)

>I think the problem has to do with the "predictive interface" that
>tries to generate the normal form of the matrix, so it uses a lot of
>memory (and even produces a kernel crash).

A few comments:

First, I cannot reproduce your results. That using your code
slightly cleaned up, I don't see any problems. By slightly
cleaned up I mean replacing K (which as a built-in meaning) with
k and eliminating the local variable res which isn't needed to get:

gmat[k_, l_, m_] := Module[{bandU, bandL, bandD},
   bandU = Table[l, {k - 1}];
   bandL = Table[m, {k - 1}];
   bandD = Table[-l - m, {k}];
   bandD[[1]] = -l;
   bandD[[k]] = -m;
   DiagonalMatrix[SparseArray[bandU], 1] +
    DiagonalMatrix[SparseArray[bandD]] +
    DiagonalMatrix[SparseArray[bandL], -1]]

And this code can be further simplified to:

gMat[k_, l_, m_] :=
  SparseArray[{Band[{1, 1}] -> -m - l, Band[{1, 2}] -> l,
     Band[{2, 1}] -> m}, {k, k}] +
   SparseArray[{{1, 1} -> m, {k, k} -> l}]

But I do see a difference between version 8 and 9 with respect
to the amount of memory used. That is:

In[1]:= gMat[k_, l_, m_] :=
  SparseArray[{Band[{1, 1}] -> -m - l, Band[{1, 2}] -> l,
     Band[{2, 1}] -> m}, {k, k}] +
   SparseArray[{{1, 1} -> m, {k, k} -> l}]

In[2]:= a = MemoryInUse[];
gMat[10000, .8, 1]; MemoryInUse[] - a

Out[3]= 1003920

In[4]:= $Version

Out[4]= 9.0 for Mac OS X x86 (64-bit) (November 20, 2012)

and

In[1]:= gMat[k_, l_, m_] :=
  SparseArray[{Band[{1, 1}] -> -m - l, Band[{1, 2}] -> l,
     Band[{2, 1}] -> m}, {k, k}] +
   SparseArray[{{1, 1} -> m, {k, k} -> l}]

In[2]:= a = MemoryInUse[];
gMat[10000, .8, 1]; MemoryInUse[] - a

Out[3]= 493024

In[4]:= $Version

Out[4]= 8.0 for Mac OS X x86 (64-bit) (October 5, 2011)

In version 9 on my system, the array requires a bit more than
twice the memory that version 8 requires. So, there may be
something to your speculation about the predictive interface.

One other thing. I should point out I have 16GB of RAM installed
which may be why I am not seeing problems with your code.




  • Prev by Date: Re: question
  • Next by Date: PolynomialQuotient slow
  • Previous by thread: Re: Large SparseArray in Mathematica 9 and the predictive interface
  • Next by thread: Re: Large SparseArray in Mathematica 9 and the predictive interface