[Date Index] [Thread Index] [Author Index]
Re: accessing data in a large file
You can use Save command (see Math Documents library) to save the intermediate data as separate files. Basically, you divide the original data set into different segments, and each segment is saved with a different name such as one, two, three..so on. Once you retrieve the entire dataset, then you can combine them as one file. Regards, Temel -----Original Message----- From: wb at wavebounce.com [mailto:wb at wavebounce.com] Sent: Tuesday, January 27, 2009 12:59 PM To: mathgroup at smc.vnet.net Subject: [mg95803] [mg95784] accessing data in a large file I'm trying to get data from a 450MB file. I can get the first 300MB (or 150 000 000 Integer16) with the code below but any more than that causes Mathematica 7 to shut down with out of memory error (WinXP, SP3, 2GB RAM). n = 150000000; inFile = OpenRead["DR.wav", BinaryFormat -> True]; y = BinaryReadList[inFile, "Integer16", n]; Close[inFile]; My reading of the documentation indicates that the only option I have with BinaryReadList is to read from the start of the file to depth n (negative values of n it won't accept). If I could start reading from some point into the file, I could split the file up into pieces by exporting segments to other files. Also, I've found nothing in the documentation that tells me how Mathematica 7 might use the HD to store some of this data as I read it in. Can anyone suggest a method (however slow) that would allow me access to all the data in this file? Thanks.