Re: Re: [long] Mathematica 5.1 and memory: any garbage collection?
- To: mathgroup at smc.vnet.net
- Subject: [mg59767] Re: [mg59727] Re: [long] Mathematica 5.1 and memory: any garbage collection?
- From: Pratik Desai <pdesai1 at umbc.edu>
- Date: Sat, 20 Aug 2005 03:14:09 -0400 (EDT)
- References: <de12lv$8ka$1@smc.vnet.net> <200508190831.EAA27431@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
Jens-Peer Kuska wrote: >Hi, > >have you tryed to clear the cache with > >Developer`ClearCache[ ] > >?? > >Regards > Jens > ><some.poster at use.net> schrieb im Newsbeitrag >news:de12lv$8ka$1 at smc.vnet.net... >| Hi, >| >| I've been using Mathematica v5.1 under XP for >some time now and found it largely >| improved over previous versions, at least for my >problems. >| >| Only recently did I begin to handle some real >data: rather large List >| objects. I have to routinely select objects >within lists as large as, say, >| 46000 * 512 * 20 items, items themselves not >being atomic. >| >| Of course I never expected to be able to use >functionnal style for dealing >| with such large guys on a 1Gb RAM PC: an outer >For loop hands one object >| (out of 46000) at a time, then functionnal style >handling follows. In this >| process I create variable-sized intermediate >objects that are either >| accumulated in decimated lists or discarded >altogether. >| >| It turns out that my typical intermediate >objects eat up to 2.5 Mb each. >| This sounds a little of a memory hog, but >anyway, I can live with that. >| >| My problem lies with the fact that memory used >at intermediate stage seems >| never be freed, even whith explicit Clear, >Remove or any assignment to {} or >| . in whatever order. >| >| How can one recover the memory freed (if some is >ever effectively freed)? Is >| there _any_ garbage collection in the guts of >this gazillion dollars piece >| of code? >| >| I have no problem with the idea of regularly >invoking some function to >| compact the heap (should the kernel be unable to >perform it by itself). The >| question is: what's its name? >| >| How do you reclaim memory, you serious users who >do computations on large >| objects with numerous intermediate values? >| >| >| At the end, I tried this, from a fresh kernel >load: >| >| In[1]:= >| $ProductInformation >| >| Out[1]= >| {ProductIDName\[Rule]Mathematica, >| ProductKernelName\[Rule]Mathematica 5.1 Kernel, >| ProductVersion\[Rule]5.1 for Microsoft Windows >(October 25, 2004), >| ProductVersionNumber\[Rule]5.1} >| >| In[2]:= >| MemoryInUse[] >| >| Out[2]= >| 2124904 >| >| In[3]:= >| (* let's create a moderate-size object *) >| try=Tuples[{a,b,c,d,e,f},7]; >| ByteCount[try] >| {MemoryInUse[],MaxMemoryUsed[]} >| >| Out[4]= >| 14556704 >| >| Out[5]= >| {16684640,16685792} >| >| In[6]:= >| (* simple mapped function *) >| res=Map[2*#&,try]; >| {MemoryInUse[],MaxMemoryUsed[]} >| >| Out[7]= >| {93949328,103186832} (* Ouch >! So much for a _simple_ test *) >| >| In[8]:= >| (* forgetting a large expression doesn't free >_any_ memory *) >| try={}; >| {MemoryInUse[],MaxMemoryUsed[]} >| >| Out[9]= >| {93951056,103186832} >| >| In[10]:= >| (* no more success this way *) >| try=.; >| {MemoryInUse[],MaxMemoryUsed[]} >| >| Out[11]= >| {93952648,103186832} >| >| In[12]:= >| (* ... neither a sweep does *) >| Clear[try]; >| {MemoryInUse[],MaxMemoryUsed[]} >| >| Out[13]= >| {93954320,103186832} >| >| In[14]:= >| (* ... nor a complete wash *) >| Remove[try]; >| {MemoryInUse[],MaxMemoryUsed[]} >| >| Out[15]= >| {93955944,103186832} >| >| In[16]:= >| (* freeing the result also seems beyond possible >*) >| res={}; >| {MemoryInUse[],MaxMemoryUsed[]} >| >| Out[17]= >| {93958112,103186832} >| >| In[18]:= >| (* still no luck, dad *) >| res=.; >| {MemoryInUse[],MaxMemoryUsed[]} >| >| Out[19]= >| {93958952,103186832} >| >| In[20]:= >| (* ahem, what _is_ Clear supposed to do? *) >| Clear[res]; >| {MemoryInUse[],MaxMemoryUsed[]} >| >| Out[21]= >| {93960448,103186832} >| >| In[22]:= >| (* ahem, what _is_ Remove supposed to do? *) >| Remove[res]; >| {MemoryInUse[],MaxMemoryUsed[]} >| >| Out[23]= >| {93961936,103186832} >| >| In[24]:= >| (* sharing common subexpressions somehow works, >slow as hell *) >| >| (* please note: even after removing all values >and symbols in all >| possible ways, try and res are _still_ >somewhere in memory since >| Share _really_ does something! *) >| >| (* Share should share nothing as we don't have >any object stored, safe >| perhaps symbols a..f *) >| >| (* when we use larger objects or more >intermediate results like res >| we get the kernel to shutdown very quickly at >this rate *) >| Share[] >| {MemoryInUse[],MaxMemoryUsed[]} >| >| Out[24]= >| 62824872 >| >| Out[25]= >| {31138184,103186832} >| >| (* did you expect to see about the same value as >on the first call above? *) >| > > > > Developer`ClearCache[ ] How does this work ? Since the help does not really provide any concrete example it is hard to implement it. Does anybody know how to implement it also any help on the following two, perhaps with an example will be greatly appreciated ClearCache["Numeric"] ClearCache["Symbolic"] Needless to say I have tried the variant of all three, to no success Best Regards Pratik -- Pratik Desai Graduate Student UMBC Department of Mechanical Engineering Phone: 410 455 8134
- References:
- Re: [long] Mathematica 5.1 and memory: any garbage collection?
- From: "Jens-Peer Kuska" <kuska@informatik.uni-leipzig.de>
- Re: [long] Mathematica 5.1 and memory: any garbage collection?