Re: Numerically computing partial derivatives
- To: mathgroup at smc.vnet.net
- Subject: [mg47962] Re: Numerically computing partial derivatives
- From: "Steve Luttrell" <steve_usenet at _removemefirst_luttrell.org.uk>
- Date: Tue, 4 May 2004 01:08:44 -0400 (EDT)
- References: <c72d7k$jk$1@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
I think you should be able to make a lot of progress analytically. I attach
an extract from one of my papers in which I compute the first derivative of
a quantity that is closely related to yours. The main tricks to use are log
det = trace log, the Baker-Hausdorff identity for expanding logs of products
of matrices, trace(commutator)=0, etc. I presume these sorts of tricks can
be used to compute the Hessian as well.
Select from (******** to *********) and paste into Mathematica.
Steve Luttrell
(************** Content-type: application/mathematica **************
CreatedBy='Mathematica 5.0'
Mathematica-Compatible Notebook
This notebook can be used with any Mathematica-compatible
application, such as Mathematica, MathReader or Publicon. The data
for the notebook starts with the line containing stars above.
To get the notebook into a Mathematica-compatible application, do
one of the following:
* Save the data starting with the line of stars above into a file
with a name ending in .nb, then open the file inside the
application;
* Copy the data starting with the line of stars above to the
clipboard, then use the Paste menu command inside the application.
Data for notebooks contains only printable 7-bit ASCII and can be
sent directly in email or through ftp in text mode. Newlines can be
CR, LF or CRLF (Unix, Macintosh or MS-DOS style).
NOTE: If you modify the data for this notebook not in a Mathematica-
compatible application, you must delete the line below containing
the word CacheID, otherwise Mathematica-compatible applications may
try to use invalid cache data.
For more information on notebooks and Mathematica-compatible
applications, contact Wolfram Research:
web: http://www.wolfram.com
email: info at wolfram.com
phone: +1-217-398-0700 (U.S.)
Notebook reader applications are available free of charge from
Wolfram Research.
*******************************************************************)
(*CacheID: 232*)
(*NotebookFileLineBreakTest
NotebookFileLineBreakTest*)
(*NotebookOptionsPosition[ 4550, 141]*)
(*NotebookOutlinePosition[ 5349, 170]*)
(* CellTagsIndexPosition[ 5274, 164]*)
(*WindowFrame->Normal*)
Notebook[{
Cell[CellGroupData[{
Cell[TextData[{
StyleBox["Differentiating",
FontSlant->"Italic"],
" log det"
}], "Title"],
Cell[TextData[{
"We differentiate the logarithm of the determinant of a \
matrix-valued quantity. We use this in order to differentiate ",
ButtonBox["equation",
ButtonData:>"Eq:LowerBoundIntegrated",
ButtonStyle->"Hyperlink"],
" (",
"XXX",
"), so we present the derivation using an appropriate notation."
}], "Text"],
Cell[BoxData[
FormBox[GridBox[{
{\(Step\ 1\), \(\[Delta]\ log[
det(\[Pi]\ \[Sigma]\^\[Prime])]\), \(\(=\)\(\(-log[
det(\[Pi](\(\[Sigma]\^\[Prime]\)\^\(-1\) + \(\
\[Delta]\[Sigma]\^\[Prime]\)\^\(-1\)))]\) +
log[det(\[Pi]\ \(\[Sigma]\^\[Prime]\)\^\(-1\))]\)\)},
{\(Step\ 2\),
" ", \(\(=\)\(\(-tr[
log(\(\(\[Sigma]\^\[Prime]\)\^\(-1\)\)(
1 + \(\[Sigma]\^\[Prime]\) \(\[Delta]\[Sigma]\
\^\[Prime]\)\^\(-1\)))]\) +
tr[log(\(\[Sigma]\^\[Prime]\)\^\(-1\))]\)\)},
{\(Step\ 3\),
" ", \(\(=\)\(-tr[
log(1 + \(\[Sigma]\^\[Prime]\) \(\[Delta]\[Sigma]\^\
\[Prime]\)\^\(-1\))]\)\)},
{\(Step\ 4\),
" ", \(\(\[TildeEqual]\)\(-\(\(tr[\(\[Sigma]\^\[Prime]\) \
\(\[Delta]\[Sigma]\^\[Prime]\)\^\(-1\)]\)\(.\)\)\)\)}
}], TraditionalForm]], "NumberedEquation",
TextAlignment->AlignmentMarker,
GridBoxOptions->{ColumnAlignments->{Left}}],
Cell["\<\
We justify the various stages of this manipulation as follows:\
\>", "Text"],
Cell[TextData[{
"Step 1. Matrix invert ",
Cell[BoxData[
\(TraditionalForm\`\[Sigma]\^\[Prime]\)]],
", which introduces a minus sign outside the logarithm function. In \
order to calculate the derivative, write the difference that results \
from changing ",
Cell[BoxData[
\(TraditionalForm\`\(\[Sigma]\^\[Prime]\)\^\(-1\)\)]],
" infinitesimally."
}], "Text"],
Cell[TextData[{
"Step 2. Use the identity ",
Cell[BoxData[
\(TraditionalForm\`log[det(X)] = tr[log(X)]\)]],
"."
}], "Text"],
Cell[TextData[{
"Step 3. Use the identity ",
Cell[BoxData[
\(TraditionalForm\`log(X\ Y) =
log(X) +
log(Y) + \((commutator\ terms\ from\ the\ Baker -
Hausdorff\ identity)\)\)]],
" to obtain ",
Cell[BoxData[
\(TraditionalForm\`tr[log(X\ Y)] = tr[log(X)] + tr[log(Y)]\)]],
" which causes a pair of terms to cancel, leaving only the \
infinitesimal part. Note that the trace of any commutator is zero."
}], "Text"],
Cell[TextData[{
"Step 4. Expand the logarithm using ",
Cell[BoxData[
\(TraditionalForm\`log(1 + X) =
X + \[ScriptCapitalO](X\^2)\)]],
"."
}], "Text",
CellTags->"Ed:Change2"]
}, Open ]]
},
FrontEndVersion->"5.0 for Microsoft Windows",
ScreenRectangle->{{0, 1280}, {0, 941}},
WindowSize->{686, 740},
WindowMargins->{{0, Automatic}, {Automatic, 0}},
StyleDefinitions -> "Report.nb"
]
(*******************************************************************
Cached data follows. If you edit this Notebook file directly, not
using Mathematica, you must remove the line containing CacheID at
the top of the file. The cache data will then be recreated when
you save this file from within Mathematica.
*******************************************************************)
(*CellTagsOutline
CellTagsIndex->{
"Ed:Change2"->{
Cell[4338, 131, 196, 7, 29, "Text",
CellTags->"Ed:Change2"]}
}
*)
(*CellTagsIndex
CellTagsIndex->{
{"Ed:Change2", 5177, 157}
}
*)
(*NotebookFileOutline
Notebook[{
Cell[CellGroupData[{
Cell[1776, 53, 97, 4, 81, "Title"],
Cell[1876, 59, 334, 9, 46, "Text"],
Cell[2213, 70, 1043, 22, 89, "NumberedEquation"],
Cell[3259, 94, 86, 2, 29, "Text"],
Cell[3348, 98, 379, 10, 46, "Text"],
Cell[3730, 110, 135, 5, 29, "Text"],
Cell[3868, 117, 467, 12, 63, "Text"],
Cell[4338, 131, 196, 7, 29, "Text",
CellTags->"Ed:Change2"]
}, Open ]]
}
]
*)
(*******************************************************************
End of Mathematica Notebook file.
*******************************************************************)