Re: [long] Mathematica 5.1 and memory: any garbage collection?
- To: mathgroup at smc.vnet.net
- Subject: [mg59727] Re: [long] Mathematica 5.1 and memory: any garbage collection?
- From: "Jens-Peer Kuska" <kuska at informatik.uni-leipzig.de>
- Date: Fri, 19 Aug 2005 04:31:49 -0400 (EDT)
- Organization: Uni Leipzig
- References: <de12lv$8ka$1@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
Hi, have you tryed to clear the cache with Developer`ClearCache[ ] ?? Regards Jens <some.poster at use.net> schrieb im Newsbeitrag news:de12lv$8ka$1 at smc.vnet.net... | Hi, | | I've been using Mathematica v5.1 under XP for some time now and found it largely | improved over previous versions, at least for my problems. | | Only recently did I begin to handle some real data: rather large List | objects. I have to routinely select objects within lists as large as, say, | 46000 * 512 * 20 items, items themselves not being atomic. | | Of course I never expected to be able to use functionnal style for dealing | with such large guys on a 1Gb RAM PC: an outer For loop hands one object | (out of 46000) at a time, then functionnal style handling follows. In this | process I create variable-sized intermediate objects that are either | accumulated in decimated lists or discarded altogether. | | It turns out that my typical intermediate objects eat up to 2.5 Mb each. | This sounds a little of a memory hog, but anyway, I can live with that. | | My problem lies with the fact that memory used at intermediate stage seems | never be freed, even whith explicit Clear, Remove or any assignment to {} or | . in whatever order. | | How can one recover the memory freed (if some is ever effectively freed)? Is | there _any_ garbage collection in the guts of this gazillion dollars piece | of code? | | I have no problem with the idea of regularly invoking some function to | compact the heap (should the kernel be unable to perform it by itself). The | question is: what's its name? | | How do you reclaim memory, you serious users who do computations on large | objects with numerous intermediate values? | | | At the end, I tried this, from a fresh kernel load: | | In[1]:= | $ProductInformation | | Out[1]= | {ProductIDName\[Rule]Mathematica, | ProductKernelName\[Rule]Mathematica 5.1 Kernel, | ProductVersion\[Rule]5.1 for Microsoft Windows (October 25, 2004), | ProductVersionNumber\[Rule]5.1} | | In[2]:= | MemoryInUse[] | | Out[2]= | 2124904 | | In[3]:= | (* let's create a moderate-size object *) | try=Tuples[{a,b,c,d,e,f},7]; | ByteCount[try] | {MemoryInUse[],MaxMemoryUsed[]} | | Out[4]= | 14556704 | | Out[5]= | {16684640,16685792} | | In[6]:= | (* simple mapped function *) | res=Map[2*#&,try]; | {MemoryInUse[],MaxMemoryUsed[]} | | Out[7]= | {93949328,103186832} (* Ouch ! So much for a _simple_ test *) | | In[8]:= | (* forgetting a large expression doesn't free _any_ memory *) | try={}; | {MemoryInUse[],MaxMemoryUsed[]} | | Out[9]= | {93951056,103186832} | | In[10]:= | (* no more success this way *) | try=.; | {MemoryInUse[],MaxMemoryUsed[]} | | Out[11]= | {93952648,103186832} | | In[12]:= | (* ... neither a sweep does *) | Clear[try]; | {MemoryInUse[],MaxMemoryUsed[]} | | Out[13]= | {93954320,103186832} | | In[14]:= | (* ... nor a complete wash *) | Remove[try]; | {MemoryInUse[],MaxMemoryUsed[]} | | Out[15]= | {93955944,103186832} | | In[16]:= | (* freeing the result also seems beyond possible *) | res={}; | {MemoryInUse[],MaxMemoryUsed[]} | | Out[17]= | {93958112,103186832} | | In[18]:= | (* still no luck, dad *) | res=.; | {MemoryInUse[],MaxMemoryUsed[]} | | Out[19]= | {93958952,103186832} | | In[20]:= | (* ahem, what _is_ Clear supposed to do? *) | Clear[res]; | {MemoryInUse[],MaxMemoryUsed[]} | | Out[21]= | {93960448,103186832} | | In[22]:= | (* ahem, what _is_ Remove supposed to do? *) | Remove[res]; | {MemoryInUse[],MaxMemoryUsed[]} | | Out[23]= | {93961936,103186832} | | In[24]:= | (* sharing common subexpressions somehow works, slow as hell *) | | (* please note: even after removing all values and symbols in all | possible ways, try and res are _still_ somewhere in memory since | Share _really_ does something! *) | | (* Share should share nothing as we don't have any object stored, safe | perhaps symbols a..f *) | | (* when we use larger objects or more intermediate results like res | we get the kernel to shutdown very quickly at this rate *) | Share[] | {MemoryInUse[],MaxMemoryUsed[]} | | Out[24]= | 62824872 | | Out[25]= | {31138184,103186832} | | (* did you expect to see about the same value as on the first call above? *) |
- Follow-Ups:
- Re: Re: [long] Mathematica 5.1 and memory: any garbage collection?
- From: Pratik Desai <pdesai1@umbc.edu>
- Re: Re: [long] Mathematica 5.1 and memory: any garbage collection?