MathGroup Archive 2000

[Date Index] [Thread Index] [Author Index]

Search the Archive

ReadList question

  • To: mathgroup at
  • Subject: [mg23731] ReadList question
  • From: ermi at (Ernie)
  • Date: Mon, 5 Jun 2000 01:09:23 -0400 (EDT)
  • Organization: Cisco Systems Inc
  • Sender: owner-wri-mathgroup at

	I have been using Mathematica 4.0 at work and it is really
helpful. I am having trouble importing fairly large datafiles within a
reasonable amount of time. The data file is about 50Mb of tab
delimited text in table format with three header rows, which comes to
420,000 lines or 550,000 words in size. 
	I run PIII 500Mhz Dual Pentuim Unit w/384Mb ram and I have
shut down all other process to accomplish these tasks. Yet no matter
how I try this it takes entirely too long, as it runs overnight and
still does not complete.
	Is there another routine (function) set I should be using? I
need to read in the data as it is, cut off the header rows, and
extract the columns with the numerical data in them for processing. I
will be needing to process files of this size and larger in the
future. Have any of you had experience with this sort of thing in

	The code I use is as such:
<< Statistics`DataManipulation`
<< Statistics`DescriptiveStatistics`

ldapResults = 
    ReadList["d:\System Test Data\Bug.Dirquery\Data.out", Word, 
      RecordLists -> True, WordSeparators -> {"\t"}];

mydatasize = Length[ldapResults]

resultsTable = Take[ldapResults, {4, mydatasize}];

fulldatasize = Length[resultsTable]

workingset = ToExpression[Column[resultsTable, {1, 3, 5, 8, 9}]]

Thanks all.

-- Ernie

  • Prev by Date: Re: system of nonlinear ODE
  • Next by Date: Re: simple question....very simple
  • Previous by thread: Re: system of nonlinear ODE
  • Next by thread: Re: ReadList question