Re: Export vs. Put; Import vs. Get
- To: mathgroup at smc.vnet.net
- Subject: [mg36563] Re: Export vs. Put; Import vs. Get
- From: "P.J. Hinton" <paulh at wolfram.com>
- Date: Wed, 11 Sep 2002 13:27:58 -0400 (EDT)
- Organization: "Wolfram Research, Inc."
- Sender: owner-wri-mathgroup at wolfram.com
On Wed, 11 Sep 2002, Mike wrote: > I'm curious about the different efficiencies of Export vs. Put; Import vs. > Get. > > Eg. Saving a list called "c" with approx. 16,000 elements, each element > being a three element list of reals. > > Export["ac.dat",c] takes forever and consumes heaps of kernel memory > > c>>ac.dat all done in an instant (relatively speaking) and without the > transient burst of kernel memory usage. > > Ditto Import and Get. Can anyone explain this? The comparison is that of apples and oranges. Get and Put deal with reading and writing Mathematica expressions in InputForm syntax. They are implemented in C, so they are fairly fast, and the target format is human readable. However the output might not be readily parsable by other computer programs. The functions may be used with Mathematica expressions of all kinds. In the event that human readability and platform independence are not important, one can use DumpSave in lieu of Put. Export and Import for the cases you describe are using the "Table" target format. They are implemented in top-level Mathematica code, so they are not as fast as Get and Put. The target is a regularly formatted array of data, so not all Mathematica expressions are appropriate for this kind of conversion. In the case of Export, the expression is formatted completely in memory using TableForm before being saved out to file. In the case of Import, there are a number of heuristics that are applied to each field to determine whether the field should be converted to a number or a string. This should explain why they are slower and require more memory. If you are dealing with a large collection of machine precision numbers, then neither of these approaches may be suitable for your purposes. You might be better off using the free add-on FastBinaryFiles, which is available up on MathSource at this URL: http://www.mathsource.com/Content/Enhancements/MathLink/0206-783 This writes out the numerical data in binary form, which is readily readable by C programs. -- P.J. Hinton User Interface Programmer paulh at wolfram.com Wolfram Research, Inc. Disclaimer: Opinions expressed herein are those of the author alone.