[Date Index] [Thread Index] [Author Index]

Re: Loading large files

• To: mathgroup at smc.vnet.net
• Subject: [mg86227] Re: Loading large files
• From: David Bailey <dave at Remove_Thisdbailey.co.uk>
• Date: Thu, 6 Mar 2008 03:00:24 -0500 (EST)
• References: <fqlmqf$jg4$1@smc.vnet.net>

Akki wrote:
> Hi,
> 
> I am trying to solve a Linear Program for which the matrix A is very
> very large.
> The size of the matrix A runs into GB (~1GB). I have broken it into
> smaller
> files of size ~100 MB each and would like to load the files in
> Mathematica (Basically copy paste them). I tried to open and paste
> into mathematica but I cannot paste beyond the first file (~100MB)
> 
> My question is can Mathematica  support such a large matrix ?
> 
> Thanks,
> Darth
> 
Pasting the data is a very bad way to get such a large volume of 
information into Mathematica. Mathematica consists of two programs - the 
frontend and the kernel. Pasting involves the frontend - which will try 
to format the result and choke (to use technical terminology!)

Depending on the format of the original data (what sort of file is it 
in), you should try the Import function, or possibly use OpenRead to 
create a stream and then use ReadList

Because of its large size, be very careful to ensure the resulting array 
is packed - see my answer to "Inadequate capacity".

David Bailey
http://www.dbaileyconsultancy.co.uk