Re: Memory Leak in Mathematica?
- To: mathgroup at smc.vnet.net
- Subject: [mg54548] Re: [mg54338] Memory Leak in Mathematica?
- From: John Fultz <jfultz at wolfram.com>
- Date: Tue, 22 Feb 2005 04:25:34 -0500 (EST)
- Reply-to: jfultz at wolfram.com
- Sender: owner-wri-mathgroup at wolfram.com
Do you know that SetDirectory[] uses a stack which can be popped by using
ResetDirectory[]?
It would take a truly stunning amount of calls for this to be a real
problem, but perhaps that's exactly what's going on in your case when, as
you say, the number of files is large. If so, then the problem is easy to
fix. Change your code from...
SetDirectory[newDirectory];
Export[filename, dataOut, "CSV"];
SetDirectory[oldDirectory];
to....
SetDirectory[newDirectory];
Export[filename, dataOut, "CSV"];
ResetDirectory[];
which would pop the old value off the stack, thereby preventing extra
memory usage.
Sincerely,
John Fultz
jfultz at wolfram.com
User Interface Group
Wolfram Research, Inc.
On Sat, 19 Feb 2005 02:31:39 -0500 (EST), M.G. Bartlett wrote:
> Folks,
>
> Has anyone ever experienced a problem with Mathematica leaking memory?
> I am running 5.1 on WindowsXP and have run into a memory issue from the
> following code which essentially just steps through a directory of
> files, reading each in and doing some processing on the data, then
> writing to a new location:
>
>
> (*Set a read directory and a write directory, get filenames to
> process*)
> oldDirectory = Directory[];
> newDirectory = "D:\\NewDirectory";
> currentFileNames = FileNames["*",{oldDirectory}];
>
> processFile[filename_] := Module[{dataIn, dataOut},
>
> (*Load the file*)
> dataIn = Import[filename, "CSV"];
>
> (*Do some work on the data in the file*)
> dataOut = Map[Drop[#, -2] &, dataIn];
>
> (*Write to the newDirectory*)
> SetDirectory[newDirectory];
> Export[filename, dataOut, "CSV"];
> SetDirectory[oldDirectory];
> ]
>
> Map[processFile, currentFileNames];
>
>
> My problem is that when the number of files to be read in is large,
> Mathematica quits mid-process and returns an "Out of Memory" message.
> However, as I read the code it should never be using more memory then
> that required to hold a single file in memory. As I watch
> Mathematica's memory usage as the process runs, however, it slowly
> consumes more and more of the machine's resources. Am I missing
> something in the code that would cause Mathematica to hold on to memory
> or is this a bug in Mathematica?
>
> Thanks,
>
> Marshall Bartlett