Re: Matrix Differentiation in Mathematica
- To: mathgroup at smc.vnet.net
- Subject: [mg23205] Re: Matrix Differentiation in Mathematica
- From: "David Bailey" <dave-bailey at freeuk.com>
- Date: Mon, 24 Apr 2000 01:12:12 -0400 (EDT)
- References: <8djkrl$7hr@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
Johannes Ludsteck <ludsteck at zew.de> wrote in message news:8djkrl$7hr at smc.vnet.net... > Dear MathGroup members, > I am wondering whether there are some Limitations to matrix- > differentiation in Mathematica. > I would like to get the partial derivative of expressions like > f[c.x], where c is a vector of constants and x a vector of variables. > I know it would be possible to do this after specifying c and x > (for example x = {x1,x2,x3}; c = {c1,c2,c3}), but > in my application it is much more elegant and efficient to do this > with c and x unspecified. > For the first derivative of this expression with respect to x > Mathematica returns the right result: > D[f[c.x],x] /. Dot[___,0,___]->0 > Out[1] = c.1 f'[c.x] > However - to the best of my knowledge - it is impossible to ask > Mathematica to compute the derivative of this expression (the first > derivative) with respect to the transpose of x. > D[%,x] /. Dot[___,0,___]->0 > gives > (c.1)^2 f''[c.x] > This cannot be the right solution since Mathematica has no > information on the dimension of x. > The right solution would be > Outer[Times,c,c] f''[c.x]. > Of course, I could correct this by hand, substituting (c.1)^2 by > Outer[Times,c,c]. But since the expression I want to differentiate is > very complicated, this would come to the same thing as doing the > computations by pencil and paper. How is Mathematica supposed to know that x is a vector in your example - OK it might 'guess' because it appears as an argument to Dot - but in reality I think it just runs its scalar derivative code. I think you need to write a few recursive rules which define your vector derivatives - say vD. You also need to decide a few things, such as whether all your functions take scalar arguments and return scalar results or whether you need to allow for more general posibilities. Also, if you take several vector derivatives you will generate some high order tensors, so I think it might be better to define vector differentiation as differentiation wrt a component of the vector. Here are enough rules to solve your case. As written it assumes it is working with scalar functions of one argument and that nothing is an implicit function of X. Note that your outer product now just ends up as a simple product of two subscripted variables. (The notation looks better if you turn it back to StandardForm :) ) Clear[vD]; vD[A_ . B_, Subscript[X_, j_]] := vD[A, Subscript[X, j]] . B + vD[B, Subscript[X, j]] . A; vD[(f_)[A_], Subscript[X_, j_]] := Derivative[1][f][A]*vD[A, Subscript[X, j]]; vD[A_*B_, Subscript[X_, j_]] := vD[A, Subscript[X, j]]*B + vD[B, Subscript[X, j]]* A; vD[Subscript[X_, k_], Subscript[X_, j_]] := KroneckerDelta[k, j]; vD[X_, Subscript[X_, j_]] := Subscript[Id, j]; vD[A_, Subscript[X_, j_]] := 0; cleanup[x_] := x //. {a_ . 0 -> 0, 0 . a_ -> 0, Subscript[Id, j_] . A_ -> Subscript[A, j], A . Subscript[Id, j_] -> Subscript[A, j]} (First derivative) cleanup[vD[g[A . X], Subscript[X, j]]] Subscript[A, j]*Derivative[1][g][A . X] (Second derivative) cleanup[vD[Subscript[A, j]*Derivative[1][g][A . X], Subscript[X, k]]] Subscript[A, j]*Subscript[A, k]*Derivative[2][g][ A . X] I hope this helps. There may, of course, be a package out there that does this sort of thing. David Bailey Salford Software db at salford-software.com