Re: Storing and Loading Definitions / Emulating Associative Arrays
- To: mathgroup at smc.vnet.net
- Subject: [mg93515] Re: Storing and Loading Definitions / Emulating Associative Arrays
- From: Nikolaus Rath <Nikolaus at rath.org>
- Date: Thu, 13 Nov 2008 04:03:36 -0500 (EST)
- References: <gf3mfg$ekk$1@smc.vnet.net> <gfefjg$n7m$1@smc.vnet.net>
David Bailey <dave at removedbailey.co.uk> writes:
> Nikolaus Rath wrote:
>> Hi,
>>
>> I have a routine that calculates some sequence of numbers. The
>> calculation takes very long, so I would like to store the results in a
>> file.
>>
>> What I would like to do is to pack the data, together with the
>> parameters of its generation, into some sort of associative array.
>> Since I couldn't find such a thing in Mathematica, I came up with the
>> idea of using definitions instead:
>>
>>
>> f[seed_, p_, n_] = (* Complicated computation *) {2,4,4};
>> data[source] = "simulation";
>> data[seed] = 42;
>> data[parameters] = {12, 0};
>> data[values] = f[data[seed], Sequence @@ data[parameters]]
>>
>> Now I would like to (somehow) store this set of definitions:
>>
>> Write[stream, something[data]]
>>
>> and later load it again, possibly with a different name
>>
>> data2 = Read[stream]
>>
>>
>> But what is the function something[] that I have to use for this
>> purpose? If I simply store the expression "data", then it is written
>> literally, without the associated definitions. If I use
>> Definition[data] then the output includes the name "data", so that I
>> cannot easily associate it with "data2" when loading. Is there a
>> simple way to achieve what I want?
>>
>> I guess a way would be to store definitions as an anonymous function:
>>
>> data = Function[{x}, Switch[x,
>> source, "simulation",
>> seed, 42,
>> parameters, {12, 0},
>> values, f[42, 12, 0]]];
>>
>> Write[stream, data]
>>
>> but then I still need a way to convert between the anonymous function
>> and the definitions when saving and loading. I do not want to work
>> with the anonymous function all the time, since it seems very
>> difficult to modify any parameters after the initial definition:
>>
>> data = Read[Stream]
>> (* do some stuff *)
>> data[comment] = "Analysis on 11/7 showed that this situation is u=
nli=
>> kely";
>>
>> (* Write back the data with the comment *)
>>
>
> Before committing to a particular scheme, I think you need to be a
> bit more specific about the size of your problem. For example, are
> you talking about thousands of numbers, or hundreds of millions. In
> the latter case, you may want to think of schemes that avoid loading
> all the data into memory at once.
No, the data I'm working with fits nicely into memory.
> Save or DumpSave may well be all you need (DumpSave creates a binary
> file which is much faster to read back in).
Yes, DumpSave (and Save) already looks quite promising. The only thing
that I don't like about it is the fact that the name of the definition
is saved as well. I would like to e.g. save the definitions of a
function f and then load them as definitions for a function g (without
affecting an already defined f). Is there a way to accomplish that?
> Remember that your final program may work fast enough without any
> special tricks, or the bottleneck may turn out to other than where
> you anticipate it to be.
I am not really creating a standalone program but using Mathematica
interactively to analyze data from a numerical simulation. Right now I
have exported the data into .csv and load it from there, but this is
quite messy so I'm looking for a better way to do it.
Best,
-Nikolaus
--
=C2=BBIt is not worth an intelligent man's time to be in the majority.
By definition, there are already enough people to do that.=C2=AB
-J.H. Hardy
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C