RE: Saving Large Matrices
- To: mathgroup at smc.vnet.net
- Subject: [mg24807] RE: [mg24773] Saving Large Matrices
- From: "Tomas Garza" <tgarza at mail.internet.com.mx>
- Date: Sun, 13 Aug 2000 03:16:45 -0400 (EDT)
- Sender: owner-wri-mathgroup at wolfram.com
bbongiorno at attglobal.net wrote:
> I need a effective way to store and retrieve a 300,000 x 44 matrix that
> includes alpha numeric and numeric cells.
>
> The data is first downloaded from a database into a mathematica variable
> in a note book. Included in some of the alpha numeric cells are commas,
> tabs and spaces and therefore I cannot use Write function. If I use the
> Write function to store the matrix, the integrity of the shape of the
> matrix is impaired.
>
> The Export function works but it takes 24 hours to store the matrix on
> a Pentium II 300 with major virtual memory usage.
Try >> (Put). This will store your matrix in Mathematica format, and you can
retrieve it with << (Get). Example:
In[11]:=
aMat = Table[Random[Integer, {0, 9}], {300000}, {44}]; // Timing
Out[11]=
{78.82 Second, Null}
(It is quite large, indeed: it takes 78 seconds just to generate a random
matrix that size). Now you store it in a file called "filaMat":
In[12]:=
aMat >> filaMat; // Timing
Out[12]=
{152.03 Second, Null}
The storage process takes 152 seconds. Now you retrieve it:
In[13]:=
<< filaMat; // Timing
Out[13]=
{159.45 Second, Null}
The retrieval process takes 159 seconds.
Tomas Garza
Mexico City