MathGroup Archive 2004

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Re: Re: No more memory available

  • To: mathgroup at smc.vnet.net
  • Subject: [mg51228] Re: [mg51201] Re: [mg51165] Re: No more memory available
  • From: Daniel Lichtblau <danl at wolfram.com>
  • Date: Sat, 9 Oct 2004 04:18:46 -0400 (EDT)
  • References: <ck0bht$ns5$1@smc.vnet.net> <200410070925.FAA10723@smc.vnet.net> <200410080654.CAA24963@smc.vnet.net>
  • Sender: owner-wri-mathgroup at wolfram.com

János wrote:
> It is stack={{{{{},d},c},b},a}.
> 
> In the meantime I measured the depth of the stack and it is normal, I 
> mean it is mostly below 100 during a run. So, I was wrong in my 
> assumption that my stack is getting too deep.   My apology...   I ran 
> out of memory, because the results - different length of strings - I 
> collect in a simple list is getting too big.
> I will try to compress or pack them before I add them to the list.
> 
> In the meantime as I investigated the Depth[of different objects], I 
> found that I can crash the kernel with no exception on both of my 
> machines with the following one liner:
> 
> Depth[Fold[List, "", Table["", {i, 87363}]]]
> 
> I looked the Book, but found no limit on nesting other than 
> $RecursionLimit, but that one does not apply here looks like anyway.  
> So the biggest depth an object can have without crashing the kernel in 
> 5.0.1 is 87361.  Interestingly it is a multiplicand of two primes 199 
> and 439.
> 
> Thanks a lot,
> 
> János
> [...]

In many places in the Mathematica code base recursion is used. Depth is 
an obvious candidate. This has the unfortunate effect of exploding the 
subroutine stack when manipulating deeply nested expressions.

Several years ago (for version 3.0.1, I believe) I fixed this for a few 
important and commonly used functions such as Flatten, LeafCount, and 
our expression freeing code, by using an explicit stack internally to 
avoid the operating system (and maybe compiler) dependent subroutine 
stack. (At least one or two other people contributed to that work so I 
don't want to claim all the credit, or blame, in case people noticed a 
slight slowdown.) Anyway, this was not done for all imaginable 
functions. Possibly we should consider handling a few more in that way. 
Offhand I'm not sure how common it is to do such things as Depth on 
deeply nested expressions.

I do not know if there is some OS-dependent significance to the fact 
that 87361 = 199*439, but clearly it had to be a product of SOME primes.


Daniel Lichtblau
Wolfram Research



  • Prev by Date: Re: webMathematica and loss of context
  • Next by Date: Re: Occurrence of a substring inside a list of strings
  • Previous by thread: Re: Re: No more memory available
  • Next by thread: COMBINATORICAL PROBLEM