[Date Index] [Thread Index] [Author Index]

Import Limitations

• To: mathgroup at smc.vnet.net
• Subject: [mg61065] Import Limitations
• From: Benedetto Bongiorno <bbongiorno at attglobal.net>
• Date: Sun, 9 Oct 2005 01:35:21 -0400 (EDT)
• Sender: owner-wri-mathgroup at wolfram.com

To Mathgroup,

I frequently need to Import large data, usually a *.csv file, containing 
3 million rows by 50 columns.
The file contains both strings and numerics. I sometimes do not know the 
make up of which columns are strings and which are numerics.
In small files I import all the data as strings.
Because of the 2 gig memory limitation of Windows XP running V5.2, I 
cannot use the Import.
The reason Import does not work for me is it will only import the 
complete file, thus the system runs
out of memory. I need to import the data in small groups.
I have tried ReadList, but it only lets you select the first group size.
Suggestions of Skip and SetStreamPosition causes me errors, due to the 
fact that I have a problem
composing the functions.

Thank you,

Ben Bongiorno