Services & Resources / Wolfram Forums / MathGroup Archive
-----

MathGroup Archive 2008

[Date Index] [Thread Index] [Author Index]

Search the Archive

SparseArray memory usage

  • To: mathgroup at smc.vnet.net
  • Subject: [mg87839] SparseArray memory usage
  • From: wpb <w.p.bergsma at lse.ac.uk>
  • Date: Fri, 18 Apr 2008 02:41:28 -0400 (EDT)

Hi,

An obvious advantage of SparseArray is that less memory is needed for sparse arrays. Unfortunately, this doesn't always seem to be the case, as I outline below sometimes almost as much memory is used as for an ordinary array.

Can someone explain this, and hopefully tell me how to get around it?

First, a situation which is fine:
Starting with a clean kernel, I get the following memory usage for a 2 x 100million sparse array:

In[1]:= MaxMemoryUsed[]
MemoryInUse[]
a=SparseArray[{{1,1}->1},{2,100000000}];
MaxMemoryUsed[]
MemoryInUse[]

Out[1]= 5826800
Out[2]= 6652184
Out[4]= 6927152
Out[5]= 6654624

Second, a situation which is not fine: I use the transposed array and memory usage explodes:

In[1]:= MaxMemoryUsed[]
MemoryInUse[]
a=SparseArray[{{1,1}->1},{100000000,2}];
MaxMemoryUsed[]
MemoryInUse[]
Out[1]= 5826816
Out[2]= 6652184
Out[4]= 806653720
Out[5]= 406655144

Thanks for any help.


  • Prev by Date: Re: Re: List concatenation - two more methods, one truly fast
  • Next by Date: Re: PolarPlot
  • Previous by thread: Re: Quantile and InverseCDF
  • Next by thread: Re: SparseArray memory usage