Re: Differentiation ?

*To*: mathgroup at smc.vnet.net*Subject*: [mg14702] Re: [mg14668] Differentiation ?*From*: Yves Gauvreau <gauy at videotron.ca>*Date*: Sun, 8 Nov 1998 21:15:58 -0500*Sender*: owner-wri-mathgroup at wolfram.com

-----Original Message----- From: Jurgen Tischer <jtischer at col2.telecom.com.co> To: mathgroup at smc.vnet.net Subject: [mg14702] Re: [mg14668] Differentiation ? >Hi Yves, >as to your first problem: > >In[1]:= s = Sum[y[i]^2 - 2 mu y[i] + mu^2, {i, 1, n}] > >In[2]:= Evaluate /@ D[s, mu] > >Out[2]= Sum[2*mu - 2*y[i], {i, 1, n}] > >You need the Evaluate/@ because Sum has Attribute HoldAll. Now if you >insist in the formula you ended up with (or better with the correct >version) you would have to write some simplification rules to apply to >the result, something like: > >In[3]:= simplifySum={ > Sum[x_+y_.,{n_,na_,ne_}]/;FreeQ[x,n]:>(ne-na+1)x+ Sum[y,{n,na,ne}], > Sum[x_ y_,{n_,na_,ne_}]/;FreeQ[x,n]:>x Sum[y,{n,ne,na}]} > >In[4]:= Evaluate/@D[s,mu]/.simplifySum > >Out[4]= 2*mu*n + Sum[-2*y[i], {i, 1, n}] > > >Now to the second problem: >First of all, I think your formula (apart from the trivial error of a >factor of 2 of the first term) is false. In any case it's a problem of >interpretation. Let me show what I mean with an example, using matrices >in M(2,2). So my interpretation of your formula E'E = Y'Y - B'X'Y + >B'X'XB is that there you have a function f[B_]:=Y'Y-B'X'Y+B'X'X B where >Y, X are constants (in M(2,2)), and you are searching the derivative. >Now by linearity of the derivative we can interchange derivation and >addition so we have: the first term is constant, so the derivative is 0, >the second term is linear (in b) so the derivative is THE SAME linear >function and the third term is quadratic, so by the rules for >derivatives of bilinear forms (see for example Dieudonne', A treatise in >analysis) we get all together D[f,B][U]==U'X'Y+U'X'X B+B'X'X U. So far >the treatment without Mathematica, now lets do it with Mathematica. We >identify M(2,2) with R^4 (just use Flatten on the matrices), so the >original function now reads > >In[1]:= >x={{x11,x12},{x21,x22}}; >y={{y11,y12},{y21,y22}}; >b={{b11,b12},{b21,b22}}; >u={{u11,u12},{u21,u22}}; > >In[2]:= f[{b11_,b12_,b21_,b22_}]= > Flatten[Transpose[y].y-Transpose[b].Transpose[x].y+ > Transpose[b].Transpose[x].x.b]; > >In[2]:= df=Outer[D,f[{b11,b12,b21,b22}],{b11,b12,b21,b22}]; > >(* This df is in M(4,4) and represents the derivative in the usual >matrix form. Lets check if the two results coincide. *) > >In[3]:= df.Flatten[u]== > Flatten[-Transpose[u].Transpose[x].y+Transpose[u].Transpose[x].x.b+ > Transpose[b].Transpose[x].x.u]//Simplify > >Out[3]= True > >This brings us back to your original question: Is it possible to >implement this type of derivation in Mathematica. As I see it, if you >need a concrete derivative of that type (in finite, low order dimension) >the method of identifying M(n,n) with R^(n^2) is viable. If you want >derivation in Banach spaces in a theoretic fashion, you would have to >implement it, and I think that would be quite a challenge (think only >how to implement the rule for multilinear functions and how to identify >a multilinear function automatically). > > >Jurgen > > > > >Yves Gauvreau wrote: >> >> Hi, >> >> I saw this equation in a book (Sum => greek SIGMA) >> >> Sum ei^2 = Sum (yi^2 - 2 mu yi + mu^2) >> >> That easy to implement but there's this one >> >> D[ Sum ei^2, mu] = 2 Sum yi - 2 n mu >> >> and this one to >> >> E'E = Y'Y - B'X'Y + B'X'XB >> >> D[E'E,B] = -2 X'Y + 2X'XB >> >> the ' mean Transpose[] >> >> How can I implement these type of Differenciation ? Is it possible to >> do that in Mathematica ? >> >> Thanks >> Yves > > I'll give you all of the author solution approch. """"""""""""""""""" This is the general linear regression model Y = XB + E eq 1 where Y is n x 1 matrix of observed values X is an n x (m+1) matrix of independent variables B is an (m-1) x 1 matrix of the unknown parameters E is an n x 1 matrix of error terms The sums of squares to be minimized is written in matrix form as E'E = (Y - XB)' (Y - XB) = Y'Y - B'X'Y + B'X'XB eq 2 To minimize this function, we take the derivative with respect to the matrix B and get the following: D[E'E,B] = -2 X'Y + 2X'XB eq 3 Equating to zero yields: (X'X)Bhat = X'Y eq 4 The solutions to this matrix equation are Bhat = inverse(X'X)X'Y eq 5 """""""""""""""""""""" As you can see the final solution is correct and as you mentionned about the -2 in equation 3 I had the same concern and that is the reason of my query. I wanted to know if I could solve this in Mathematica and arrive at the same result. Due to the way M does thing I found that I could not setup the problem litteraly and I hoped someone had an idea on how to do it. Thanks Yves