Re: Limits to Det Function

*To*: mathgroup at yoda.physics.unc.edu*Subject*: Re: Limits to Det Function*From*: David Withoff <withoff>*Date*: Thu, 23 Apr 1992 18:49:14 -0500

Ross Moore write: > The recent discussion concerning symbolic determinants has overlooked a > very significant constraint, imposed by Mathematica. That is, how large can > an array/list be? ... > Conclusion: with lists of size > 2^20 (approx) don't expect even the > simplest operations to work. Break up the array into smaller pieces and > work with these, if you can. > > Is this apparent size limit machine-dependent ? and/or version-dependent ? > Would someone please confirm (or rebuff) my suspicions here. The time required for manipulating large lists is dependent on the amount of memory available, the amount of memory used so far by the program, and details of how the operating system allocates memory. In addition to being both machine-dependent and history-dependent, it is also version dependent, since the way Mathematica uses memory has changed between versions. An experiment (on a NeXT): In[1]:= Timing[a = Range[2^20];] Out[1]= {17.1667 Second, Null} In[2]:= Timing[b = Range[2^20];] Out[2]= {20.8333 Second, Null} In[3]:= Timing[c = Range[2^20];] Out[3]= {24.2 Second, Null} In[4]:= Timing[d = Range[2^20];] Out[4]= {26.5333 Second, Null} In[5]:= Timing[e = Range[2^20];] Out[5]= {30.5333 Second, Null} In[6]:= Clear[a,b,c,d,e] In[7]:= Timing[a = Range[2^20];] Out[7]= {14.1833 Second, Null} As the machine looks harder and harder to find memory, the timing gets longer and longer. When the symbols holding all the memory are cleared, the time is faster than it was to start with, since Mathematica kept track of where the free memory was. Also, the real (wall clock) time is far greater than shown here, particularly for Out[5], since most of the time was probably spent rerranging the disk, and not included in the result of Timing. The only built in limits to the size of expressions are around 2^32 -- far larger than anything considered here. Today's computers have about 2^25 bytes of memory available for this sort of process (give or take a factor of 10). Expressions like numbers use about 2^4 or 2^5 bytes of memory, so memory limitations set in for expressions with about 2^20 or 2^21 elements. Dave Withoff withoff at wri.com