MathGroup Archive 2008

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Storing and Loading Definitions / Emulating Associative Arrays

  • To: mathgroup at
  • Subject: [mg93515] Re: Storing and Loading Definitions / Emulating Associative Arrays
  • From: Nikolaus Rath <Nikolaus at>
  • Date: Thu, 13 Nov 2008 04:03:36 -0500 (EST)
  • References: <gf3mfg$ekk$> <gfefjg$n7m$>

David Bailey <dave at> writes:
> Nikolaus Rath wrote:
>> Hi,
>> I have a routine that calculates some sequence of numbers. The
>> calculation takes very long, so I would like to store the results in a
>> file.
>> What I would like to do is to pack the data, together with the
>> parameters of its generation, into some sort of associative array.
>> Since I couldn't find such a thing in Mathematica, I came up with the
>> idea of using definitions instead:
>> f[seed_, p_, n_] = (* Complicated computation *) {2,4,4};
>> data[source] = "simulation";
>> data[seed] = 42;
>> data[parameters] = {12, 0};
>> data[values] = f[data[seed], Sequence @@ data[parameters]]
>> Now I would like to (somehow) store this set of definitions:
>> Write[stream, something[data]]
>> and later load it again, possibly with a different name
>> data2 = Read[stream]
>> But what is the function something[] that I have to use for this
>> purpose? If I simply store the expression "data", then it is written
>> literally, without the associated definitions. If I use
>> Definition[data] then the output includes the name "data", so that I
>> cannot easily associate it with "data2" when loading. Is there a
>> simple way to achieve what I want?
>> I guess a way would be to store definitions as an anonymous function:
>>     data = Function[{x}, Switch[x,
>>        source, "simulation",
>>        seed, 42,
>>        parameters, {12, 0},
>>        values, f[42, 12, 0]]];
>>     Write[stream, data]
>> but then I still need a way to convert between the anonymous function
>> and the definitions when saving and loading. I do not want to work
>> with the anonymous function all the time, since it seems very
>> difficult to modify any parameters after the initial definition:
>>    data = Read[Stream]
>>    (* do some stuff *)
>>    data[comment] = "Analysis on 11/7 showed that this situation is u=
>> kely";
>>    (* Write back the data with the comment *)
> Before committing to a particular scheme, I think you need to be a
> bit more specific about the size of your problem. For example, are
> you talking about thousands of numbers, or hundreds of millions. In
> the latter case, you may want to think of schemes that avoid loading
> all the data into memory at once.

No, the data I'm working with fits nicely into memory.

> Save or DumpSave may well be all you need (DumpSave creates a binary
> file which is much faster to read back in).

Yes, DumpSave (and Save) already looks quite promising. The only thing
that I don't like about it is the fact that the name of the definition
is saved as well. I would like to e.g. save the definitions of a
function f and then load them as definitions for a function g (without
affecting an already defined f). Is there a way to accomplish that?

> Remember that your final program may work fast enough without any
> special tricks, or the bottleneck may turn out to other than where
> you anticipate it to be.

I am not really creating a standalone program but using Mathematica
interactively to analyze data from a numerical simulation. Right now I
have exported the data into .csv and load it from there, but this is
quite messy so I'm looking for a better way to do it.



 =C2=BBIt is not worth an intelligent man's time to be in the majority.
  By definition, there are already enough people to do that.=C2=AB
                                                         -J.H. Hardy

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C

  • Prev by Date: Re: Storing and Loading Definitions / Emulating Associative Arrays
  • Next by Date: Re: Re: Re: How do little quickest
  • Previous by thread: Re: Storing and Loading Definitions / Emulating Associative Arrays
  • Next by thread: Re: Storing and Loading Definitions / Emulating Associative Arrays