Mathematica 9 is now available
Student Support Forum
-----
Student Support Forum: 'huge file' topicStudent Support Forum > General > "huge file"

< Previous Comment | Next Comment >Help | Reply To Comment | Reply To Topic
Author Comment/Response
matt prior
09/21/07 09:03am

Hi there,

you might find this is a task best accomplished outside of mathematica.

unix/linux provides a robust command line text sorting tool, aptly named 'sort'. It's capable of handling quite large amounts of data .... I've not tried it on a 5GB file but you could incrementally
test that it worked on larger and larger file sizes.
If you aren't running on linux .. you can install the free cygwin package under windows, it has an implementation of sort you could use.


If you find that the 'sort' approach doesn't work
you might want to investiagte implmentations of the merge sort algorithm which will work when a lof your data to be sorted will not fit in the memory on your computer.



All the best,
Matt

URL: ,

Subject (listing for 'huge file')
Author Date Posted
huge file Andrey 09/20/07 8:10pm
Re: huge file matt prior 09/21/07 09:03am
Re: Re: huge file Andrey 09/24/07 8:17pm
< Previous Comment | Next Comment >Help | Reply To Comment | Reply To Topic