Recommended to Wolfram Research
- To: mathgroup at yoda.physics.unc.edu
- Subject: Recommended to Wolfram Research
- From: Scott Perkins <perkins at acs00.cc.vanderbilt.edu>
- Date: Sun, 10 May 92 01:21:22 CDT
>Date: Wed, 6 May 1992 09:52:07 -0500
>From: "Theodore W. Gray" <guam!me at wri.com>
>To: perkins at acs00.cc.vanderbilt.edu
>Subject: Stack overflow
>
>
>As for things Apple could or could not do to improve performance,
>
>there are a lot. For one thing, the virtual memory swapping speed in
>System 7 is very slow. For another, having to decide ahead of time
>how much memory to allow Mathematica to ever use tends to cause
>people to allot less memory that is going to be needed. If
>
>Mathematica could grow its partition as needed, you would be much
>less likely to run out of memory. Not that this excuses crashing on
>out of memory condition, which should never happen, but still it
>would make you much more likely to get an answer. (Even if we were
>all perfect superhumans, Mathematica would still not be able to give
>a correct answer without enough memory.)
>
Thanks for the response. I have a few suggestions you may want to test:
HEAP CONSIDERATIONS:
Under System 7 it is possible for Mathematica to use memory beyond the minimum
partition established at launch time (provided that there is extra memory
available). Apple calls this temporary memory, since it is not permanently
reserved for your application. This still doesn't solve the problem of having a
fixed size stack, but it might help avoid the need to shut down the program to
adjust it's memory. This can be annoying if you create an animation which takes
a long time and then you discover you need more memory and have to run it again
from the beginning.
Another good idea would be to make the graphics purgeable into a temporary disk
file. Having to keep the whole notebook in memory is a limitation on notebook
size, program speed, and general functionality, and will cause lots of segment
swapping and VM swapping, of which you have complained about the speed. No VM
system can match the efficiency of the programmer who intelligently purges
unneeded resources.
STACK CONSIDERATIONS:
I feel that reliability is more important than speed. You may want to poll the
user community to see how they feel about this. It would be very impressive if
Mathematica were the only symbolic math product on the market that NEVER
crashed.
In any case it would be better if Mathematica had some kind of stack checking of
it's own, since the MacOS kills Mathematica the moment the stack overflows.
There is no chance to save your work after a stack overflow. I have a
recommendation on a way to implement this that would not slow things down very
much.
MacOS tracks stack overflows by a sniffer mechanism that runs 60 times per
second. This sniffer is a simple routine that compares the stack limit address
with the current stack pointer. If the pointer is greater than the limit you
get the bomb box.
You can use the same approach. Make a determination about what would be a good
amount of "cushion" space in your stack. Then check your stack pointer during
the main event loop of Mathematica to see if it has entered the cushion range.
The amount of cushion would be based on how fast the stack can fill up between
calls to the main loop and this depends on the speed of the CPU to some extent,
but also on the design of Mathematica's internal multi-tasking. If you can
guarantee a limited number of recursions between calls to the scheduler
(something you must already be doing), then your home-free. You will have
protection and you are not making the check before every recursive call. To
really make this plan work the stack cushion may have to be pretty big, say 256K
or so. but that is a small amount of memory to trade in return for reliability.
Wouldn't it be nice to be able to set recursion limit to infinity without
worrying about a crash? Then you could recurse to your heart's content and if
you run out of space it's no big deal.
Another user on the list sent me a piece of sample code that demonstrates this:
bikini [65] roach> math20
Mathematica 2.0 for NeXT
Copyright 1988-91 Wolfram Research, Inc.
-- NeXT graphics initialized --
In[1]:= $RecursionLimit=Infinity;$IterationLimit=Infinity;
In[2]:= Foo[n_]:=
Block[{},
Foo[n+1]]
In[3]:= Foo[1]
/usr2/users/roach/math20: 842 Memory fault
bikini [66] roach>
Notice that this was run on the NeXT, not the Macintosh. Though this example
was deliberate, it's just not right for the user to be able to type something
that makes the program crash.
At Vanderbilt we have many crashes that occur when a working function is re-run
using larger arguments. One professor was demonstrating to a class how the
output of a function changes based on the input arguments. He tried a larger
argument each time, but finally a stack overflow occurred. He was using a
MacIIfx with 32 MB of real RAM, with Mathematica 2.0 using about 16 MB or so. I
won't get into the details because this bug report has already been addressed.
But I appeal to you to recognize the need for a better approach to stack useage.
Thanks for listening,
Scott