Import Limitations
- To: mathgroup at smc.vnet.net
- Subject: [mg61065] Import Limitations
- From: Benedetto Bongiorno <bbongiorno at attglobal.net>
- Date: Sun, 9 Oct 2005 01:35:21 -0400 (EDT)
- Sender: owner-wri-mathgroup at wolfram.com
To Mathgroup, I frequently need to Import large data, usually a *.csv file, containing 3 million rows by 50 columns. The file contains both strings and numerics. I sometimes do not know the make up of which columns are strings and which are numerics. In small files I import all the data as strings. Because of the 2 gig memory limitation of Windows XP running V5.2, I cannot use the Import. The reason Import does not work for me is it will only import the complete file, thus the system runs out of memory. I need to import the data in small groups. I have tried ReadList, but it only lets you select the first group size. Suggestions of Skip and SetStreamPosition causes me errors, due to the fact that I have a problem composing the functions. Thank you, Ben Bongiorno