Re: How to kill slave kernel securely?
- To: mathgroup at smc.vnet.net
- Subject: [mg117445] Re: How to kill slave kernel securely?
- From: Oliver Ruebenkoenig <ruebenko at wolfram.com>
- Date: Sat, 19 Mar 2011 05:16:49 -0500 (EST)
On Fri, 18 Mar 2011, Alexey Popkov wrote:
> Oliver Ruebenkoenig <ruebenko at wolfram.com> wrote:
>> On Thu, 10 Mar 2011, Alexey Popkov wrote:
>>> 2) My bitter experience with MemoryConstrained clearly shows that it is
> not
>>> a reliable function. It practical cases when I heavily use such
> functions as
>>> NDSolve, NIntegrate etc. it often allows MathKernel to take for example
>>> 500-900 MB more that I have set. And system goes into swapping...
>>
>> You could forward that to the tech support for us to look at.
>>
>> Oliver
>
> I have found today a simple example when MemoryConstrained fails:
>
> In[1]:= MaxMemoryUsed[]
> MemoryConstrained[
> Quiet@Module[{f, n = 0},
> f[x_] := f[x + 1] + Plus @@ Table[Random[], {10^7}];
> Block[{$RecursionLimit = 3000}, f[0]]];, 90000000];
> MaxMemoryUsed[]
>
> Out[1]= 10757752
>
> Out[3]= 291258232
>
> You can see that MathKernel takes more than 260 Mb additional memory in peak
> although only less than 90 Mb additional memory was allowed by
> MemoryConstrained. This behavior is machine-dependent. Try to play with the
> memory constrain if you cannot reproduce.
Actually I do not see that:
Mathematica 8.0 for Linux x86 (64-bit)
Copyright 1988-2011 Wolfram Research, Inc.
In[1]:= MaxMemoryUsed[]
Out[1]= 13993528
MemoryConstrained[
Quiet@Module[{f, n = 0},
f[x_] := f[x + 1] + Plus @@ Table[Random[], {10^7}];
Block[{$RecursionLimit = 3000},
f[0]]];, 90000000]; MaxMemoryUsed[]
Out[2]= 14521744
For version 8 the memory allocator has been rewritten and improved.
(What a small sentence for such a huge endeavor and such a fine execution)
Here is another example with a lower bound
MaxMemoryUsed[]/1024.^2
MemoryConstrained[
Quiet@Module[{f, n = 0},
f[x_] := f[x + 1] + Plus @@ Table[Random[], {10^8}];
Block[{$RecursionLimit = 20000}, f[0]]];, 20000000];
MaxMemoryUsed[]/1024.^2
>
> As I said before, my experience shows that in practice the overshoot can be
> 900 Mb and more! This often happens in complicated computations with heavy
> usage such functions as NDSolve and NIntegrate inside loops.
>
> Alexey
>
>
There may be various issues here, for example, NDSolve calls external
libraries. And M- does not and can not have control over those memory
processes. Similar issues may arise with e.g. MKL.
Could you provide an example, where the overshot is 900 Mb - assuming you
see this in V8.
Oliver