Re: Exporting and importing arbitrary expressions efficiently
- To: mathgroup at smc.vnet.net
- Subject: [mg60075] Re: [mg60048] Exporting and importing arbitrary expressions efficiently
- From: Chris Chiasson <chris.chiasson at gmail.com>
- Date: Thu, 1 Sep 2005 02:13:10 -0400 (EDT)
- References: <200508300843.EAA05774@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
I don't know how to solve the Import problem, but wanted to remind you
that you can still keep all of your commands in Mathematica if you use
its ability to execute shell commands from within the notebook.
On 8/30/05, J. McKenzie Alexander <jalex at lse.ac.uk> wrote:
> Mathematica 5.1 says it has highly optimized import and export of
> binary data, plus automated encoding and decoding of .gz files. If I
> have Mathematica-generated data that consists of complex lists of
> (effectively) arbitrary expressions, what is the best way to save
> this to a data file and read that data file back in later?
>
> The best solution I've found for *exporting* is to use Export, saving
> to a .m.gz file. For example,
>
> > In[1]:=
> > <<DiscreteMath`Combinatorica`
> >
> > In[2]:=
> > Timing[
> > Export["~/filetest.m.gz",
> > Table[{RandomGraph[10,.3],Table[Random[],{25}]},{1000}]
> > ]
> > ]
> >
> > Out[2]=
> > {2.63919 Second,~/filetest.m.gz}
>
> I don't know if that's the highly optimized export that was touted,
> but it's certainly perfectly fine. But what's the "highly optimized"
> way to *import* this data? Using Import is painfully slow. For
> example,
>
> > In[3]:=
> > Timing[
> > data=Import["~/filetest.m.gz"];
> > ]
> >
> > Out[3]=
> > {41.3851 Second,Null}
>
> If I gunzip the file by hand, Get pulls the file in about 80 times
> *faster* than import.
>
> > In[4]:=
> > Timing[
> > data1=Get["~/filetest.m"];
> > ]
> >
> > Out[4]=
> > {0.500501 Second,Null}
>
> Granted, Get doesn't have to gunzip the file, but there's no way that
> should take so long. Timing gunzip on the same file from the command
> line says it takes only a fraction of a second:
>
> > evolve: ~
> > $time gunzip filetest.m.gz
> >
> > real 0m0.049s
> > user 0m0.025s
> > sys 0m0.015s
>
> Any pointers on what I'm doing wrong would be appreciated. I'm
> running v 5.1 on Mac OS X 10.4.
>
> Cheers,
>
> Jason
> --
> J. McKenzie Alexander
> Department of Philosophy, Logic and Scientific Method
> London School of Economics and Political Science
> Houghton Street, London WC2A 2AE
>
>
>
--
Chris Chiasson
http://chrischiasson.com/
1 (810) 265-3161