Re: Error on importing previously saved data file: Get::bigfile
- To: mathgroup at smc.vnet.net
- Subject: [mg122386] Re: Error on importing previously saved data file: Get::bigfile
- From: Tomas Garza <tgarza10 at msn.com>
- Date: Thu, 27 Oct 2011 06:30:02 -0400 (EDT)
- Delivered-to: l-mathgroup@mail-archive0.wolfram.com
Maybe you could use Put (>>) and Get(<<) instead of Save. E.g.
In[1]:= SetDirectory[$TemporaryDirectory];
In[7]:= data = RandomChoice[Range[0, 9], 1000000];
In[8]:= data >> "digits";
In[9]:= ByteCount[data]
Out[9]= 4000168
In[10]:= << digits; // AbsoluteTiming
Out[10]= {2.8584000, Null}
Takes less than 3 seconds to load, so perhaps this approach might be useful. Of course your file is about 150 times as large.
Put by default writes expressions in InputForm.
--Tomas
> Date: Wed, 26 Oct 2011 17:36:04 -0400
> From: mseise at gmail.com
> Subject: Error on importing previously saved data file: Get::bigfile
> To: mathgroup at smc.vnet.net
>
> I'm working in Mathematica 8, 64 bit.
>
> I have saved a data set (Save["filename",data]) - which is about 3.5
> MB on the harddisk (a table with numbers and images,
> ByteCount=610117226).
> When I try to read it again into Memory I get "Get::bigfile: "The file
> "filename" is too large to be read into memory."
>
> I can work around it by splitting the data in smaller junks, but there
> should be a better way since the data set should not be too large
>
> Any ideas?
>
>
> Best regards,
> Matthias Seise
>