Mathematica 9 is now available
Student Support Forum
Student Support Forum: 'Maximization Speed' topicStudent Support Forum > General > Archives > "Maximization Speed"

Help | Reply To Topic
Author Comment/Response
Randy Silvers
03/29/07 06:30am

I'm perplexed by Mathematica's behavior in numerically solving for a maximum, in the attached notebook.

In the attached, I randomly generate some data, which only takes a minute or less, and have created some functions to run regressions. These regressions are on dependent variables that are limited; consequently, Regress and NonlinearFit do not apply.

Each regression is solved by maximizing the log-likelihood function of the data. For the first, the dependent variable is binary, but for the second, it can take on one of three values, 0, 1, or 2.

In both cases, NMaximize fails to find a solution.

FindMaximum does find a solution. Additionally, taking the first derivatives and setting equal to 0, then solving this system using FindRoot also works. I've verified the accuracy by checking the same data in another program (Eviews) and obtaining the same coefficient estimates.

The problem is that whereas Eviews solves these equations in a second or two, Mathematica takes several minutes, as many as 20 or more using the fastest algorithm.

Noticing this disparity, I tried using Newton, QuasiNewton, Gradient, and ConjugateGradient; each time, Newton is by far the fastest.

Interestingly, when I reduced the WorkingPrecision->10, Mathematica solved the binary equation in about 10 seconds, which is a time that I can live with.

However, when trying the same technique on the other equation, where the dependent variable can be either 0 or 1 or 2, the speed does not improve. It still takes more than 22 minutes, and reducing the WorkingPrecision->3 produces an error after about 18 minutes.

The difference in timings cannot possibly be due to different machines. As I intend to run an experiment that will involve 20 or so runs of 1000 replications, each involving two of these estimations with the binary and two with the ordered variables -- which is why I cannot use Eviews -- it is imperative that I get the computation time down below a minute or so. The prospect of having the computer run 40000 FindMaximum commands at 20 minutes per command, well, 13000 hours is a bit long.

That, and the mystery that Mathematica, which is far more powerful than Eviews, seems much weaker. And the mystery that one technique works in one situation, but in a different situation, there is really no improvement.

The attached notebook provides the commands. The Preliminaries section processes in under 10 seconds, the Do Loop section in under two seconds. The remaining two sections show the different methods tried and you can observe the different times for each algorithm and command.

I'm including the data generation process just in case there is something amiss in there that is causing this extreme slowness, though I suspect not since Eviews would encounter the same problem.

Any assistance is greatly appreciated in saving thousands of hours!!!

Attachment: Prob-OrdProb.nb, URL:,
Help | Reply To Topic