MathGroup Archive 2006

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Re: Re: Re: "No more memory available" -- a recurring problem

  • To: mathgroup at smc.vnet.net
  • Subject: [mg68453] Re: Re: [mg68423] Re: [mg68388] Re: [mg68356] "No more memory available" -- a recurring problem
  • From: "Charlie Brummitt" <cbrummitt at wisc.edu>
  • Date: Sat, 5 Aug 2006 03:46:58 -0400 (EDT)
  • References: <200608020924.FAA28575@smc.vnet.net> <D6C89613-F0EB-4D7E-9E29-2FFAA6FB40D4@yale.edu> <321730f50608021216o11d69435g5d278a6b7d06dfa5@mail.gmail.com> <200608031007.GAA15786@smc.vnet.net> <200608040759.DAA01108@smc.vnet.net> <4035DD91-055A-434E-A2F4-AC6F14A296CB@yale.edu> <321730f50608040955g5ca1e5bdq1c40d6f76aead70d@mail.gmail.com> <6ED37A1E-916F-4F36-9639-1632859206F4@yale.edu>
  • Sender: owner-wri-mathgroup at wolfram.com

Eureka! I figured it out. In short, I had the NDSolve MaxSteps option
set to Infinity, which was causing the "No more memory available"
errors. By reducing this option to 20,000 and wrapping Check[] around
the instances in which I use NDSolve, I seem to have solved the
problem.

A heartfelt thanks to all those who have helped me solve this problem
-- János Lobb, Bill Rowe, James Gilmore, and Bruce Miller. Your
thoughtful comments and persistence uplifted me from some dark days. I
can't thank you enough.

Now, finally, back to the research...

Cheers,

Charlie

On 8/4/06, János <janos.lobb at yale.edu> wrote:
> Charlie,
>
> I would set $RecirsionLimit and its cousins to Infinity.  You might
> also hit a built in limit of Mathematica that is not obvious and
> might be processor dependent.  /I managed to to that with GraphPlot
> and the G5 duo Core :) /
>
> Are you able to run other Mathematica programs on the MacBook Pro
> fine ?  /Like creating a huge array that would eat up minimum half a
> GB memory/.  If yes, then you have to look your program with a fine
> tooth comb and nail it down where does it fail.  You might able to do
> it with a group of Print statements placed at the right places in
> your program.  If you can nail down what particular statement is
> offending the MacBook Pro, then you will have a good chance to fix it.
>
> If it dies in the middle of an NDsolve, then try that statement with
> different parameters if you can or try to program it out with a
> replacement/approximation.
>
> You might want to look the IMTEK Mathematica Supplement (IMS)
>
> http://www.imtek.de/simulation/
>
> where they provide some other methods to solve PDE-s and use their
> packages to see if those fail or not.
>
> Good luck,
>
> János
>
>
>
>
> On Aug 4, 2006, at 12:55 PM, Charlie Brummitt wrote:
>
> > János,
> >
> > Below are my responses.
> >
> >> Do you have any Mathematica.crash.log or Mathkernel.crash.log file in
> >> your ~Library/Logs/Crashreporter folder ?
> >> If yes, does the datetime entries coincide with the kernel crashes ?
> >
> > I don't see any Mathematica crash logs in my
> > ~Library/Logs/Crashreporter folder (only one file called
> > mds.crash.log, which I doubt is Mathematica-related).
> >
> >> Did you try to eliminate or cut back on History /$HistoryLength /?
> >
> > I had set $HistoryLength to 5 all along. Since learning that
> > $HistoryLength can take up a lot of memory, I set it to zero, but I
> > still get the "No more memory available" message often.
> >
> >> Did you set any iteration related env variables Wolfram provide to
> >> Infinity like $IterationLimit and $RecusionLimit, etc....
> >
> > No, I did not change any of those from the default. I rarely, if ever,
> > get other error messages, such as "reached the recursion limit,"
> > preceding the "No more memory available" message.
> >
> > Thanks again for all your help, János.
> >
> > Charlie
> >
> >
> >> On Aug 4, 2006, at 3:59 AM, Charlie Brummitt wrote:
> >>
> >> > I took my MacBook Pro into the university computer support & repair
> >> > office, and they concluded that the RAM is fine and that it
> >> probably
> >> > is not a hardware problem. I ran memtest (www.memtestosx.org/)
> >> on the
> >> > new memory stick alone (it's from OMNI Technology) and found no
> >> > errors.
> >> >
> >> > That leaves me with three possibilities:
> >> > 1) my code can simply be too difficult for a personal computer and
> >> > some sort of grid or supercomputer is required
> >> > 2) there is a bug in Mathematica, in the act of porting
> >> Mathematica to
> >> > Intel (i.e. to being universal), or in OSX which causes memory
> >> > failures even when there is plenty of memory and virtual memory
> >> > available
> >> > 3) my code is poorly written and needs radical change. To answer
> >> Bill
> >> > Rowe's question, I would consider myself an intermediate
> >> Mathematica
> >> > user, and I think I'm fairly well-versed in the area of
> >> Mathematica in
> >> > which I'm working (namely, solving and analyzing differential
> >> > equations).
> >> >
> >> > Basically, my calculation consists of using NDSolve thousands of
> >> times
> >> > to solve and evaluate PDE's in order to find ones that are chaotic.
> >> >
> >> > I'm not sure what else to do. Contact Wolfram maybe?
> >> >
> >> > I guess this ends my research project.
> >> >
> >> > Thanks to János and Bill for their helpful comments.
> >> >
> >> > Charlie
> >> >
> >> > On 8/3/06, János <janos.lobb at yale.edu> wrote:
> >> >> Charlie,
> >> >>
> >> >> Your MacBook Pro has an Intel  core duo processor and looks to me
> >> >> that Mathematica is using just one of the cores.  I do not know
> >> what
> >> >> You can do about it.  Probably nothing.  If you can put your
> >> hand on
> >> >> a Personal Grid edition that would engage the second core more
> >> >> vigorously.
> >> >>
> >> >> I do not see that you have a memory problem as regarding the
> >> usage of
> >> >> the memory reported by top.
> >> >>
> >> >> You might  have a bad memory chip and that is crashing your
> >> >> program.   Unfortunately we see bad memory coming even from Apple
> >> >> these days with newer Macs.  There is a service CD that came with
> >> >> your MacBook Pro.  Try to run the memory test from it all night
> >> in a
> >> >> loop and see if it is caching anything.
> >> >>
> >> >> You might also want to do a hard drive check with Diskwarrior.
> >> Wooly
> >> >> disks can lead to wooly virtual memory and causing crashes.
> >> >>
> >> >> Look if you have anything in ~/Library/Logs/Crashreporter
> >> regarding
> >> >> Mathematica.crash.log or Mathkernel.cash.log.   /I have plenty in
> >> >> mine for both :)/  If you find any, send them to
> >> support at wolfram.com
> >> >> and ask their opinion.
> >> >>
> >> >> I also recommend to do additional exhausting tests for the
> >> >> motherboard and other elements of the machine by using the service
> >> >> CD.
> >> >>
> >> >> In summary it looks like a hardware problem - unless you have
> >> other
> >> >> long running Mathematica programs which run just fine.
> >> >>
> >> >> With the best,
> >> >>
> >> >> János
> >> >>
> >> >> On Aug 2, 2006, at 3:16 PM, Charlie Brummitt wrote:
> >> >>
> >> >>> János,
> >> >>>
> >> >>> Thanks for the advice. I'm not really sure how to interpret the
> >> >>> information in the terminal window, so I have attached
> >> >>> screenshots of
> >> >>> it at various stages (see the names of the images). I have
> >> attached
> >> >>> them in chronological order.
> >> >>>
> >> >>> It seems to me that Mathematica is only being allocated 50% of
> >> >>> the CPU
> >> >>> and 100 megabytes of RAM! When I did the "top" on my 12"
> >> PowerBook,
> >> >>> 93% of the CPU was being used while Mathematica was running my
> >> code.
> >> >>> How do I change this on my 15" MacBook Pro?
> >> >>>
> >> >>> My 12" PowerBook is able to run this code overnight and very
> >> rarely
> >> >>> gives the "ran out of memory" error message. I would like to
> >> be able
> >> >>> to run my code overnight on my MacBook Pro, but it can only run
> >> >>> for 5
> >> >>> to 10 minutes before it runs out of memory and completely halts.
> >> >>> Thus
> >> >>> I can't "set it and leave it"; instead, I must always be at my
> >> >>> computer to restart the calculation every 5 to 10 minutes. Any
> >> >>> suggestions?
> >> >>>
> >> >>>
> >> >>> Thanks for all your help, János.
> >> >>>
> >> >>> Charlie
> >> >>>
> >> >>> On 8/2/06, János <janos.lobb at yale.edu> wrote:
> >> >>>>
> >> >>>> On Aug 2, 2006, at 5:24 AM, Charlie Brummitt wrote:
> >> >>>>
> >> >>>>> I keep getting the following error message while running a
> >> search
> >> >>>>> of many PDE's:
> >> >>>>>
> >> >>>>> "No more memory available.
> >> >>>>> Mathematica kernel has shut down.
> >> >>>>> Try quitting other applications and then retry."
> >> >>>>>
> >> >>>>> This message is becoming so common that it is crippling my
> >> >>>>> research
> >> >>>>> project, which is to find the simplest PDE with one quadratic
> >> >>>>> nonlinearity that is chaotic.
> >> >>>>>
> >> >>>>> I have Googled and searched the Mathgroup archives for help,
> >> and I
> >> >>>>> employed the following fixes:
> >> >>>>>
> >> >>>>> --CODE & SOFTWARE--
> >> >>>>> 1) Share[] -- does not help because my code rarely has common
> >> >>>> elements
> >> >>>>> that Share[] could consolidate.
> >> >>>>>
> >> >>>>> 2) I used Module[] and made as many variables local as
> >> possible. I
> >> >>>>> eliminated extra variables and functions in addition to adding
> >> >>>> Clear[]
> >> >>>>> in several places in the code to clear variables that are no
> >> >>>>> longer
> >> >>>>> needed.
> >> >>>>>
> >> >>>>> 3) I streamlined and optimized the code in general, and I
> >> made the
> >> >>>>> routines as simple and least data-intensive as possible.
> >> >>>>>
> >> >>>>> 4) I never have any other applications open when running
> >> >>>> Mathematica.
> >> >>>>> (Unfortunately, closing other applications is the only
> >> >>>> suggestion that
> >> >>>>> Mathematica provides in the "out of memory" error message.)
> >> >>>>>
> >> >>>>> 5) I should note that I run the latest version of Mathematica
> >> >>>>> (5.2.2.0).
> >> >>>>>
> >> >>>>> --HARDWARE--
> >> >>>>> 1) I upgraded from a 12" PowerBook to a 15" MacBook Pro. This
> >> >>>> computer
> >> >>>>> has an 2.16 GHz Intel Core Duo processor.
> >> >>>>>
> >> >>>>> 2) I upgraded the RAM on this new computer from 1 GB to 2 GB.
> >> >>>>>
> >> >>>>> 3) I now have much more free hard drive space (20 GB) in case
> >> >>>>> Mathematica needs to use virtual memory. In addition, this hard
> >> >>>> drive
> >> >>>>> is 7200 rpm versus the standard 5400 rpm.
> >> >>>>>
> >> >>>>>
> >> >>>>> I have been using functions like MemoryInUse[] and On
> >> >>>> [MemoryConserve]
> >> >>>>> to monitor the use of memory. I almost always find that I am
> >> only
> >> >>>>> using a few megabytes at a time (usually 5-10 MB, sometimes as
> >> >>>> high as
> >> >>>>> 100 or once 1000 after a large computation). However, despite
> >> >>>> finding
> >> >>>>> that I usually use only a few megabytes during my
> >> computations, I
> >> >>>>> often get the above "out of memory" error message, and I can
> >> never
> >> >>>>> figure out why and how much memory was needed in that
> >> particular
> >> >>>>> computation.
> >> >>>>>
> >> >>>>>
> >> >>>>> Some questions:
> >> >>>>> 1) How can I tell whether Mathematica is using virtual
> >> memory or
> >> >>>> not?
> >> >>>>> It seems to me that it is not. I have 20 GB of hard drive space
> >> >>>> free
> >> >>>>> and it never seems to use it. Why should I ever run out of
> >> >>>> memory if I
> >> >>>>> have 20 GB free in case it needs more memory than my RAM
> >> provides?
> >> >>>>>
> >> >>>>> 2) Is there a way to determine how much of my 2 GB of RAM are
> >> >>>>> allocated to Mathematica? Can I increase how much RAM
> >> >>>> Mathematica can
> >> >>>>> access?
> >> >>>>>
> >> >>>>> Thanks in advance for any help you may be able to provide! I
> >> >>>> greatly
> >> >>>>> appreciate it. If you need more information on my code or the
> >> >>>>> computations I am doing, I would be happy to provide it.
> >> >>>>>
> >> >>>>> Regards,
> >> >>>>> Charlie
> >> >>>>
> >> >>>> If you open a Terminal window on the side and type "top" and hit
> >> >>>> return and start up your application, what do You see for real
> >> >>>> memory
> >> >>>> usage for the kernel and the front end from top?
> >> >>>>
> >> >>>> János
> >> >>>> <before mathematica.jpeg>
> >> >>>> <mma loaded, code loaded, pre-execution.jpeg>
> >> >>>> <running1.jpeg>
> >> >>>> <running2.jpeg>
> >> >>>> <running3.jpeg>
> >> >>>> <running4.jpeg>
> >> >>>> <ran out of memory.jpeg>
> >> >>
> >> >>
> >>
> >>
>
>
>
> ----------------------------------------------
> Trying to argue with a politician is like lifting up the head of a
> corpse.
> (S. Lem: His Master Voice)
>
>


  • Prev by Date: Re: Re: Re: Finding the Number of Pythagorean Triples below a bound
  • Next by Date: Re: Re: Re: Finding the Number of Pythagorean Triples below a bound
  • Previous by thread: Re: Re: Re: Re: "No more memory available" -- a recurring problem
  • Next by thread: Re: "No more memory available" -- a recurring problem