Re: Working with huge text files with mathematica
- To: mathgroup at smc.vnet.net
- Subject: [mg56463] Re: [mg56378] Working with huge text files with mathematica
- From: yehuda ben-shimol <bsyehuda at gmail.com>
- Date: Tue, 26 Apr 2005 01:33:25 -0400 (EDT)
- References: <200504230516.BAA03095@smc.vnet.net>
- Reply-to: yehuda ben-shimol <bsyehuda at gmail.com>
- Sender: owner-wri-mathgroup at wolfram.com
I wonder under which operating system you are working. From the limitations you mentioned I assume that it is MS Windows which has a 2GB limitation (currently), and I suspect that writing the data into the file was done gradually and not in once. I'm I'm right in my speculations then working with unix/linux can solve it. I was working (in the past) with a Linux system using 990 MB out of 1GB RAM and 3.9GB out of 4.0 GB of swap file and the system was stable for weeks with no porblems. yehuda On 4/23/05, Andrey Shevchuk <AShevchuk at maplebank.com> wrote: > > Hi Everybody, > > I face a problem when try to read in a huge data file with my Mathematica5.1 > > The file is approx. 4Gb large and was created by another Mathematica > application (So, I think Mathematica should be able to handle it). > > Now, if I try to use Read on a stream from this file and it returns > EndOfFile and nothing else! > I checked the file with the HexEdit (an editor for huge files) and it is not > corrupt and has actually the data I need. > > A similar file (from the same application) but 1Gb works perfectly. > Any ideas? > > Does Mathematica have an internal limit for the file size (and if yes, can > one somehow override this option) ? > > I would appreciate any feedback! > >
- References:
- Working with huge text files with mathematica
- From: Andrey Shevchuk <AShevchuk@Maplebank.com>
- Working with huge text files with mathematica