Memory Management Issue - Possible Bug or Incompatibility w W7?
- To: mathgroup at smc.vnet.net
- Subject: [mg104395] Memory Management Issue - Possible Bug or Incompatibility w W7?
- From: telefunkenvf14 <rgorka at gmail.com>
- Date: Fri, 30 Oct 2009 02:18:49 -0500 (EST)
First off, my machines both run Mathematica 7.0.1 on Windows 7 Professional x64. My tablet pc has 4GB memory and my desktop has 6GB. Last week I was trying to work with some raw .txt files from the Census. The size of a typical .txt file ~15meg. When I tried using Import, Mathematica would quickly grind to a hault. After some searching in the user group about memory management tricks (such as ClearSystemCache and setting $HistoryLength=0) and using streams, I was able to get my assignment done. I used the following code to get around the problem: o = OpenRead["file"]; s = Skip[o, Record, skipto]; l = ReadList[o, Record, RecordLists -> True]; m = MatrixForm[l][[All, 1 ;; year]]; Export["newfile"]; Close[o]; Even though this worked, I STILL noted that memory usage reported in W7 was extremely high (and yes, I had restarted the kernal before applying the workaround). Anyways, this week I've been trying to work on some audio files for an EE assignment, and continue to suffer from the same general issue. As an example, one file I want to work with is a 16-bit .wav, 40 seconds in length, of an 'exponential chirp'. File size around 3.5meg. On a fresh startup, executing Import["chirp.wav","data"] eats nearly 75% of my 4GB of memory!! Below is a link to a screenshot of importing, followed by MemoryInUse, MaxMemoryUsed, and a ByteCount  estimate for a calculation that I had been trying perform previously (using streams rather than Import; same issue). Note the W7 system memory usage reported (3,154meg) vs. what Mathematica says (~64meg) is being used. http://blackboard.unl.edu/bbcswebdav/xid-1016570_1 And the notebook, if you're interested: http://blackboard.unl.edu/bbcswebdav/xid-1016571_1 Might this be caused by Wolfram's file indexing service? Other ideas? -RG