Services & ResourcesWolfram ForumsMathGroup Archive

[Date Index] [Thread Index] [Author Index]

Re: A bug in DesignedRegress?

• To: mathgroup@smc.vnet.net
• Subject: [mg10701] Re: [mg10652] A bug in DesignedRegress?
• From: seanross@worldnet.att.net
• Date: Fri, 30 Jan 1998 04:24:36 -0500
• References: <199801270810.DAA01359@smc.vnet.net.>

Osmo Kolehmainen wrote:
> 
> Hi Mathgroup!
> I am currently doing very much regression analysis,where the design
> matrix X is of size 4928 x k (k  > 26). The usual formula
> betaHat=Inverse[Transpose[X].X].Transpose[X].y works fine.
> DesignedRegress[X,y] says very soon "Out of memory", which I think is
> impossible. I am using version 3.0 for Windows 95.
> 
> Osmo Kolehmainen
> Preferable email address:
> Osmo.Kolehmainen@Joensuu.Fi


Mathematica 3.0 is not very memory efficient when it comes to storing
large arrays of data.  I would not be surprised that it did run out of
memory.  Supposing you had a 4928 x 35 array of double precision real
numbers(8 bytes each).  That is 1.4 Megabytes of data in internal
storage for one array.  Now make temporary copies of it to manipulate
it and set aside lots of memory for all the intermediate steps created
while figuring the inverse and checking for all the things a typical
mathematica internal function object does and it is certainly
believable you could run out of memory.

If you are really curious, you could look at the code in the Statistics
package to see the difference between your formula and the one that
runs out of memory.  Mathematica routines are usually very robust and
do lots of checking for special cases.  I bet that it is this that
causes the routine to run out of memory.

If you like, send me a compressed copy of your code and data and I will
run it.  My win95 system has 96M of RAM and a 500MB permanent swap
file. -- 
Remove the _nospam_ in the return address to respond.

• References:
  • A bug in DesignedRegress?
    • From: Osmo Kolehmainen <koleh@joyl.joensuu.fi>