RE: MatrixTemplate (was SpecialMatrix)

*To*: mathgroup at smc.vnet.net*Subject*: [mg28451] RE: [mg28414] MatrixTemplate (was SpecialMatrix)*From*: "Wolf, Hartmut" <Hartmut.Wolf at t-systems.de>*Date*: Thu, 19 Apr 2001 03:26:39 -0400 (EDT)*Sender*: owner-wri-mathgroup at wolfram.com

Dear Allan, dear Carl, these are most miraculous discoveries you both made. The Question arises, what makes CreateMatrix such fast? and what did you really get? a fully laid-out matrix? This is not so, I believe. For my investigation please see below... > -----Original Message----- > From: Allan Hayes [mailto:hay at haystack.demon.co.uk] To: mathgroup at smc.vnet.net > Sent: Tuesday, April 17, 2001 5:54 AM > To: mathgroup at smc.vnet.net > Subject: [mg28451] [mg28414] MatrixTemplate (was SpecialMatrix) > > > The problem of creating a matrix with all the entries the > same, say all > zero, has cropped up in recent postings. > The following seems to be very quick way of doing this: > > Clear["`*"] > CreateMatrix[x_,m_,n_]:= > PadRight[{#},m,{#}]&[PadRight[{x},{n},{x}]] > > Check > > CreateMatrix[0,3,5] > > {{0,0,0,0,0},{0,0,0,0,0},{0,0,0,0,0}} > > Timings > > CreateMatrix[0,5000,5000];//Timing > CreateMatrix[0,5000,2500];//Timing > CreateMatrix[0,2500,5000];//Timing > CreateMatrix[0,5000,250];//Timing > CreateMatrix[0,250,5000];//Timing > > {0. Second,Null} > > {0.11 Second,Null} > > {0. Second,Null} > > {0.06 Second,Null} > > {0.16 Second,Null} > > For comparison: > > IdentityMatrix[5000];//Timing > > {10.27 Second,Null} > > and. using Table, > > Clear["`*"] > CreateMatrix[x_,m_,n_]:= > Table[Evaluate[Table[x,{n}]],{m}] > > CreateMatrix[0,3,5] > > {{0,0,0,0,0},{0,0,0,0,0},{0,0,0,0,0}} > > CreateMatrix[0,5000,5000];//Timing > CreateMatrix[0,5000,2500];//Timing > CreateMatrix[0,2500,5000];//Timing > CreateMatrix[0,5000,250];//Timing > CreateMatrix[0,250,5000];//Timing > > {10.22 Second,Null} > > {5. Second,Null} > > {4.67 Second,Null} > > {0.44 Second,Null} > > {0.55 Second,Null} > > -- > Allan > --------------------- > Allan Hayes > Mathematica Training and Consulting > Leicester UK > www.haystack.demon.co.uk > hay at haystack.demon.co.uk > Voice: +44 (0)116 271 4198 > Fax: +44 (0)870 164 0565 > > > > [Hartmut Wolf] In[1]:= $Version Out[1]= "4.0 for Microsoft Windows (April 21, 1999)" In[2]:= mem0 = MemoryInUse[] Out[2]= 1129872 In[3]:= k0 = 80554 K ; ...this is the memory used as read off Microsoft Task Manager at my computer (Windows 2000). Now we go through a sequence of steps, which are obvious: In[4]:= m1 = PadRight[{#}, 2000, {#}] &@PadRight[{}, 3000, 0]; // Timing Out[4]= {0.01 Second, Null} In[5]:= mem1 = MemoryInUse[] Out[5]= 1158200 In[6]:= k1 = 80532 K; In[7]:= m2 = PadRight[{{}}, {2000, 3000}, 0]; // Timing Out[7]= {1.752 Second, Null} In[8]:= mem2 = MemoryInUse[] Out[8]= 25217776 In[9]:= k2 = 104100 K; In[10]:= m3 = Table[0, {2000}, {3000}]; // Timing Out[10]= {2.965 Second, Null} In[11]:= mem3 = MemoryInUse[] Out[11]= 49221952 In[12]:= k3 = 127560 K; ...just for comparison, a Matrix of same size: In[13]:= m4 = Partition[Range[2000*3000], 3000]; // Timing Out[13]= {0.17 Second, Null} In[14]:= mem4 = MemoryInUse[] Out[14]= 73225392 In[15]:= k4 = 151024 K; Now let's look at the memory used (per element): In[16]:= Subtract @@@ Partition[{mem4, mem3, mem2, mem1, mem0}, 2, 1]/(2000.*3000.) Out[16]= {4.00057, 4.0007, 4.00993, 0.00472133} We see m2, m3, m4 use up four bytes/element each, which is what we expect. However m1 is much more compact! In[17]:= Subtract @@@ Partition[{k4, k3, k2, k1, k0}, 2, 1] Out[17]= {23464 K, 23460 K, 23568 K, -22 K} This is memory consumption as read off the MS TaskManager. This is roughly consistent with recorded MemoryInUse. In[18]:= MemoryInUse[] Out[18]= 73231408 In[19]:= kk0 = 151412 K; Now we work with our "objects" In[20]:= m3 == m4 // Timing Out[20]= {0. Second, False} It takes no time, to find out the discrepancy, ok. In[21]:= MemoryInUse[] Out[21]= 73234112 In[22]:= kk1 = 151424 K; ...and the comparison takes no memory, ok. However... In[23]:= m1 == m4 // Timing Out[23]= {4.797 Second, False} ... the comparison with m1 takes up more time, than was gained in first place! Not ok! In[24]:= MemoryInUse[] Out[24]= 73236368 In[25]:= kk2 = 275144 K; ...what happend here? Mathematica doesn't show additional memory used, however much had been reclaimed from the OS! In[26]:= m2 == m4 // Timing Out[26]= {8.372 Second, False} In[27]:= MemoryInUse[] Out[27]= 73238624 In[28]:= kk3 = 275144 K; ..also much time for m2 needed! In[29]:= m1 == m2 // Timing Out[29]= {0.521 Second, True} In[30]:= MemoryInUse[] Out[30]= 73240896 ...this is fast In[31]:= kk4 = 275144 K; In[32]:= m2 == m3 // Timing Out[32]= {5.217 Second, True} In[33]:= MemoryInUse[] Out[33]= 73243144 In[34]:= kk5 = 275544 K; In[35]:= m1 == m3 // Timing Out[35]= {4.887 Second, True} In[36]:= MemoryInUse[] Out[36]= 73245496 In[37]:= kk6 = 275168 K; I admit that these last timings are not fully conclusive, and might be somewhat dependend on the structure of the hardware used (This is a notebook computer from my company: hp OmniBook 4150, and possibly there is a performance step down at 128 MB due to chip set, furthermore there could be some influence of paging for the last operations (paging times are not recorded by Timing[] though), my real memory was 192 MB). I will refrain from fantasizing about what is going on, whowever someone familiar with "Mathematica inside" is asked to comment. Esp. to the question: what must be paid for "rapid matrix creation", and when? Kind regards, Hartmut