MathGroup Archive 1999

[Date Index] [Thread Index] [Author Index]

Search the Archive

RE: Re: accessing large files

  • To: mathgroup at
  • Subject: [mg15886] RE: [mg15842] Re: [mg15707] accessing large files
  • From: "Jean-Marie THOMAS" <jmthomas at>
  • Date: Wed, 17 Feb 1999 23:33:36 -0500
  • Sender: owner-wri-mathgroup at

I performed some tests on huge files (two dimensional array of random
numbers), and this is a short summury:
On a Win95 system with 32M ram, handling a 6M file is a maximum, since using
Put or DumpSave takes an incredibal long time.
On a Winnt4 with 128M ram, a 10M file is okay, but saving times tend to

So it mostly depends on the ram you have. If your ram is not enough, your
operations might never end!

Hope this helps,

Jean-Marie THOMAS
mailto:jmthomas at
Conseil et Audit en Ingenierie de Calcul
Strasbourg, France

-----Original Message-----
From: Tomas Garza [mailto:tgarza at]
To: mathgroup at
Subject: [mg15886] [mg15842] Re: [mg15707] accessing large files

Chuck McCallum wrote:

>Is there a way to read in parts of very large files (in my case about
>130 megs) using mathematica? I'm trying SetStreamPosition, but it gives
>me an error when I go to read in a byte, because I'm reading past the
>end of the buffer.


My naive advice is that you use an appropriate database manager --
something like Microsoft Access --, which can easily handle files that
size (I have used it without any hitch with precisely that size files)
in order to select and prepare those parts of your files which you want
to be read on any given session, and save them as text with tab
delimiters, which you can then read into Mathematica with ReadList.
This is straightforward and painless.

Good luck,

Tomas Garza
Mexico City

  • Prev by Date: Re: List manipulation question
  • Next by Date: Figure legend & Overlapping text pb
  • Previous by thread: Re: accessing large files
  • Next by thread: thx gentleman!