Services & Resources / Wolfram Forums
-----
 /
MathGroup Archive
1999
*January
*February
*March
*April
*May
*June
*July
*August
*September
*October
*November
*December
*Archive Index
*Ask about this page
*Print this page
*Give us feedback
*Sign up for the Wolfram Insider

MathGroup Archive 1999

[Date Index] [Thread Index] [Author Index]

Search the Archive

RE: Re: accessing large files

  • To: mathgroup at smc.vnet.net
  • Subject: [mg15886] RE: [mg15842] Re: [mg15707] accessing large files
  • From: "Jean-Marie THOMAS" <jmthomas at agat.net>
  • Date: Wed, 17 Feb 1999 23:33:36 -0500
  • Sender: owner-wri-mathgroup at wolfram.com

I performed some tests on huge files (two dimensional array of random
numbers), and this is a short summury:
On a Win95 system with 32M ram, handling a 6M file is a maximum, since using
Put or DumpSave takes an incredibal long time.
On a Winnt4 with 128M ram, a 10M file is okay, but saving times tend to
increase.

So it mostly depends on the ram you have. If your ram is not enough, your
operations might never end!

Hope this helps,


****************************************
Jean-Marie THOMAS
mailto:jmthomas at agat.net
Conseil et Audit en Ingenierie de Calcul
Strasbourg, France
http://www.agat.net
****************************************

-----Original Message-----
From: Tomas Garza [mailto:tgarza at mail.internet.com.mx]
To: mathgroup at smc.vnet.net
Subject: [mg15886] [mg15842] Re: [mg15707] accessing large files



Chuck McCallum wrote:

>Is there a way to read in parts of very large files (in my case about
>130 megs) using mathematica? I'm trying SetStreamPosition, but it gives
>me an error when I go to read in a byte, because I'm reading past the
>end of the buffer.

Chuck,

My naive advice is that you use an appropriate database manager --
something like Microsoft Access --, which can easily handle files that
size (I have used it without any hitch with precisely that size files)
in order to select and prepare those parts of your files which you want
to be read on any given session, and save them as text with tab
delimiters, which you can then read into Mathematica with ReadList.
This is straightforward and painless.

Good luck,

Tomas Garza
Mexico City




  • Prev by Date: Re: List manipulation question
  • Next by Date: Figure legend & Overlapping text pb
  • Previous by thread: Re: accessing large files
  • Next by thread: thx gentleman!