MathGroup Archive 2005

[Date Index] [Thread Index] [Author Index]

Search the Archive

Import Limitations


To Mathgroup,

I frequently need to Import large data, usually a *.csv file, containing 
3 million rows by 50 columns.
The file contains both strings and numerics. I sometimes do not know the 
make up of which columns are strings and which are numerics.
In small files I import all the data as strings.
Because of the 2 gig memory limitation of Windows XP running V5.2, I 
cannot use the Import.
The reason Import does not work for me is it will only import the 
complete file, thus the system runs
out of memory. I need to import the data in small groups.
I have tried ReadList, but it only lets you select the first group size.
Suggestions of Skip and SetStreamPosition causes me errors, due to the 
fact that I have a problem
composing the functions.

Thank you,

Ben Bongiorno


  • Prev by Date: Re: Select from list
  • Next by Date: Re: Select from list
  • Previous by thread: Re: Language vs. Library
  • Next by thread: Re: Import Limitations