Mathematica 9 is now available
Services & Resources / Wolfram Forums
-----
 /
MathGroup Archive
2005
*January
*February
*March
*April
*May
*June
*July
*August
*September
*October
*November
*December
*Archive Index
*Ask about this page
*Print this page
*Give us feedback
*Sign up for the Wolfram Insider

MathGroup Archive 2005

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Working with huge text files with mathematica

  • To: mathgroup at smc.vnet.net
  • Subject: [mg56436] Re: Working with huge text files with mathematica
  • From: Maxim <ab_def at prontomail.com>
  • Date: Tue, 26 Apr 2005 01:32:38 -0400 (EDT)
  • References: <d4clv2$357$1@smc.vnet.net>
  • Sender: owner-wri-mathgroup at wolfram.com

On Sat, 23 Apr 2005 05:20:34 +0000 (UTC), Andrey Shevchuk  
<AShevchuk at Maplebank.com> wrote:

>
> Hi Everybody,
>
> I face a problem when try to read in a huge data file with my  
> Mathematica5.1
>
> The file is approx. 4Gb large and was created by another Mathematica
> application (So, I think Mathematica should be able to handle it).
>
> Now, if I try to use Read on a stream from this file and it returns
> EndOfFile and nothing else!
> I checked the file with the HexEdit (an editor for huge files) and it is  
> not
> corrupt and has actually the data I need.
>
> A similar file (from the same application) but 1Gb works perfectly.
> Any ideas?
>
> Does Mathematica have an internal limit for the file size (and if yes,  
> can
> one somehow override this option) ?
>
> I would appreciate any feedback!
>
>
>

Yes, Mathematica 5.1.0 handles files larger than 2Gb incorrectly (at least  
under Windows XP). Read and BinaryRead always return EndOfFile and  
SetStreamPosition[stream, pos] with pos!=0 just generates  
SetStreamPosition::stmrng warning. At the same time, it is possible to  
write more than 2Gb of data to a file opened with OpenWrite or  
OpenAppend...which will make the file unreadable.

Storing the data in several smaller files seems to be the only possible  
workaround.

Maxim Rytin
m.r at inbox.ru


  • Prev by Date: Re: Working with huge text files with mathematica
  • Next by Date: Re: Mathematica 5.0 notebooks icon gone.
  • Previous by thread: Re: Working with huge text files with mathematica
  • Next by thread: Re: Working with huge text files with mathematica