Creation of temporary copies of arrays when using x[[a]]=b
- To: mathgroup at smc.vnet.net
- Subject: [mg80869] Creation of temporary copies of arrays when using x[[a]]=b
- From: andrew.j.moylan at gmail.com]
- Date: Tue, 4 Sep 2007 06:35:41 -0400 (EDT)
Please consider this input, which creates a large array of real numbers, and
then changes one element of that array, and tracks the memory used by this
process:
$HistoryLength = 0;
Print[{MemoryInUse[], MaxMemoryUsed[]}];
Module[
{x},
x = RandomReal[1, {1000000}];
Print[{MemoryInUse[], MaxMemoryUsed[]}];
x[[2345]] = 5.6;
Print[{MemoryInUse[], MaxMemoryUsed[]}];
];
Print[{MemoryInUse[], MaxMemoryUsed[]}];
Here is the output (on my machine):
{5812072,5837352}
{13812760,13812920}
{13812776,21813128}
{5810712,21813128}
The second line of output shows that the code "x = RandomReal[1,
{1000000}];" used 8MB of memory, as expected for 1000000 real numbers.
The third line of output shows that the code x[[2345]]=5.6 uses *another*
8MB of memory, temporarily. Presumably this is because x[[a]]=b is
equivalent to x=ReplacePart[x,a->b], which would result in the temporary
creation of the 8MB object ReplacePart[x,a->b].
Question 1. Can anyone confirm that these are equivalent?
Because of this, an algorithm that relies on repeatedly changing single
elements of a large list by using x[[a]]=b is very slow.
Question 2. What is the simplest way to have lists for which the
reassignment of a single element is efficient? Is an external package
necessary, or can the built-in Mathematica functions be used for this
purpose?
I note that if you take the input and enclose it in Compile[{},...][], then
then output becomes:
{5840208,5864960}
{13840296,13840440}
{13840296,13840648}
{13840296,13840648}
No temporary 8MB array is created in this case. However, I can only make
this work when x is defined *inside* the compiled function. If I move the
definition of x outside Compile[], then temporary objects are created again.
Question 3. Is there a more comprehensive set of documentation for Compile
than that found in the Mathematica documentation centre? I would like to
know the answer to questions like: How does Compile decide what assumptions
to make about the types of expressions found in the code to be compiled?