MathGroup Archive 2002

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Export vs. Put; Import vs. Get

  • To: mathgroup at smc.vnet.net
  • Subject: [mg36563] Re: Export vs. Put; Import vs. Get
  • From: "P.J. Hinton" <paulh at wolfram.com>
  • Date: Wed, 11 Sep 2002 13:27:58 -0400 (EDT)
  • Organization: "Wolfram Research, Inc."
  • Sender: owner-wri-mathgroup at wolfram.com

On Wed, 11 Sep 2002, Mike wrote:

> I'm curious about the different efficiencies of Export vs. Put; Import vs.
> Get.
> 
> Eg. Saving a list called "c" with approx. 16,000 elements, each element
> being a three element list of reals.
> 
> Export["ac.dat",c] takes forever and consumes heaps of kernel memory
> 
> c>>ac.dat all done in an instant (relatively speaking) and without the
> transient burst of kernel memory usage.
> 
> Ditto Import and Get. Can anyone explain this?

The comparison is that of apples and oranges.

Get[] and Put[] deal with reading and writing Mathematica expressions in
InputForm syntax.  They are implemented in C, so they are fairly fast, and
the target format is human readable.  However the output might not be
readily parsable by other computer programs.  The functions may be used
with Mathematica expressions of all kinds.  In the event that human
readability and platform independence are not important, one can use
DumpSave[] in lieu of Put[].

Export[] and Import[] for the cases you describe are using the "Table" 
target format.  They are implemented in top-level Mathematica code, so 
they are not as fast as Get[] and Put[].  The target is a regularly 
formatted array of data, so not all Mathematica expressions are 
appropriate for this kind of conversion.  In the case of Export[], the 
expression is formatted completely in memory using TableForm[] before 
being saved out to file.  In the case of Import[], there are a number of 
heuristics that are applied to each field to determine whether the field 
should be converted to a number or a string.  This should explain why they 
are slower and require more memory.

If you are dealing with a large collection of machine precision numbers, 
then neither of these approaches may be suitable for your purposes.
 
You might be better off using the free add-on FastBinaryFiles, which is 
available up on MathSource at this URL:

http://www.mathsource.com/Content/Enhancements/MathLink/0206-783

This writes out the numerical data in binary form, which is readily 
readable by C programs.

-- 
P.J. Hinton
User Interface Programmer                         paulh at wolfram.com
Wolfram Research, Inc.
Disclaimer: Opinions expressed herein are those of the author alone.



  • Prev by Date: Re: executing a notebook from the command line
  • Next by Date: 3D chart from ASCII
  • Previous by thread: Export vs. Put; Import vs. Get
  • Next by thread: Using Slide View in mathematica 4.2