Re: Characteristic Polynomials and Eigenvalues

*To*: mathgroup at smc.vnet.net*Subject*: [mg19401] Re: Characteristic Polynomials and Eigenvalues*From*: Martin Kraus <i-martin at wolfram.com>*Date*: Mon, 23 Aug 1999 13:57:15 -0400*Organization*: Wolfram Research, Inc.*References*: <7pl6uv$cg9@smc.vnet.net>*Sender*: owner-wri-mathgroup at wolfram.com

Hi Manuel! MAvalosJr at aol.com wrote: > > Gentlemen: > > I have been studying linear algebra and with the aid of several programs and > add- ons to Mathematica the task has been a piece of cake. However, the time > comes when suddenly "understanding" leers its ugly head. > Given the vectors {4,-6}, {3, -7}, the characteristic polynomial is x^2 + 3 x > -10. The eigenvalues are (-5,2), the eigenvectors are (2,3) and (3,1). My > question: Just to be a little more explicit here is the Mathematica code: In[1]:= matrix = {{4, -6}, {3, -7}} Out[1]= {{4, -6}, {3, -7}} In[2]:= Eigenvectors[matrix] Out[2]= {{2, 3}, {3, 1}} In[3]:= Eigenvalues[matrix] Out[3]= {-5, 2} The characterisitc polynomial is calculated this way: In[4]:= Det[matrix - x*IdentityMatrix[2]] Out[4]= \!\(\(-10\) + 3\ x + x\^2\) > What does the characteristic polynomial (since it describes a curve) have to > do with the vectors (which are straight lines)? Good question. I have no idea. :) However, I know what the characteristic polynomial is good for! Why is it interesting to look at that determinant anyway? The reason is this matrix equation: matrix.v == x*v with the unknown vector v and the unknown scalar x. Let's use {v1,v2} instead of v and solve this equation with Mathematica: In[12]:= Solve[matrix.{v1, v2} == x {v1, v2}, {v1, v2, x}] Solve::"svars": "Equations may not give solutions for all \"solve\" \ variables." Out[12]= \!\({{v1 -> \(2\ v2\)\/3, x -> \(-5\)}, {v1 -> 3\ v2, x -> 2}, {v1 -> 0, v2 -> 0}}\) *wow* It works! Mathematica returns the eigenvalues for x and the eigenvectors for {v1, v2}! :) In fact the equation above defines the eigenvectors and the eigenvalues. Thus, if we map a matrix on one of its eigenvectors we get the eigenvector multiplied with the corresponding eigenvalue. In other words, eigenvectors do not change their direction but only their magnitude when the corresponding linear mapping is applied to them. Well, but that's kind of off-topic, I wanted to explain what the characteristic polynomial is. Let's modify our matrix equation by subtracting x*v on both sides: matrix.v - x*v == 0 and now rewrite this to (matrix - x*IdentityMatrix[2]).v == 0 Thus, we are close to understanding why Det[matrix - x*IdentityMatrix[2]] is an interesting polynomial! In fact, it is one result of linear algebra, that any matrix equation m.v==0 with a given matrix m and an unknown vector v has non-trivial solutions for v if Det[m]==0. Thus, it is not really the polynomial Det[matrix - x*IdentityMatrix[2]] which is interesting but its roots! Let's have a look at them: In[15]:= Roots[Det[matrix - x *IdentityMatrix[2]] == 0, x] Out[15]= x == -5 || x == 2 Ah, again the eigenvalues! And that's the only meaning of the characteristic polynomial I am aware of: Its roots are the eigenvalues of the corresponding matrix. (Note: the complete set of roots of a polynomial define it completely.) > Or for that matter, the > eigenvalues and eigenvectors -derived from the matrix or the polynomial have > to do with the vectors? > I plotted the polynomial but can't figure out what it has to do with the > vectors. > > Thanks for whatever > Manuel OK, I have explained what the relation between the characterisitc polynomial and the eigenvalues is. Also, we have seen the relation between eigenvalues and eigenvectors. So, what is the relation between the two row vectors of the matrix and the eigenvectors? Again: I have no idea! However, I know how to relate the column vectors of our matrix with the linear mapping represented by that matrix. (And we have already seen how the eigenvectors are related to the linear mapping.) The relation is quite simple: The column vectors are the images of the canonical basis vectors, i.e. applying a matrix on the canonical basis vectors returns the column vectors. In[16]:= matrix.{1, 0} Out[16]= {4, 3} and {4, 3} is the first column vector: In[17]:= Transpose[matrix][[1]] Out[17]= {4, 3} The same is true for the second column vector: In[18]:= matrix.{0, 1} Out[18]= {-6, -7} In[19]:= Transpose[matrix][[2]] Out[19]= {-6, -7} Thus, the images of the canonical basis vectors are the column vectors of a matrix, which represents a linear mapping, which has the feature that it does not change the direction of some vectors, which are called eigenvectors. However, this linear mapping does change the magnitude of these eigenvectors by factors called eigenvalues, which turn out to be the roots of a polynomial called characteristic polynomial. Can we draw a picture of this stuff? Sure: In[20]:= << Graphics`PlotField` In[21]:= << Graphics`Arrow` In[22]:= PlotVectorField[matrix.{x, y}, {x, -6, 4}, {y, -7, 3}, Epilog -> {RGBColor[0, 0, 1], Arrow[{1, 0}, matrix.{1,0}], Arrow[{0, 1}, matrix.{0,1}], RGBColor[0, 1, 0], Arrow[{0, 0}, Eigenvectors[matrix][[1]]], Arrow[{0, 0}, Eigenvectors[matrix][[2]]]}, PlotPoints -> {11, 11}, Axes -> True]; The small black arrows show the matrix: The corresponding linear mapping is applied on each position vector of the grid and the result is shown as a small black vector starting at each of the grid positions. However, the length of the resulting vectors is reduced by a common factor, such that the vectors do not intersect. For the points {1,0} and {0,1} the blue arrows correspond to the small black vectors at their starting points but with the correct length. (Obviously the plot would be a mess if all the black vectors had the correct length!) But the blue arrows given by matrix.{1,0} and matrix.{0,1} are equal to the column vectors! Thus, thinking of the small black vectors at {1,0} and {0,1} as the images of the canonical basis vectors, we see that at least their direction is equal to the direction of the column vectors which are represented by the blue arrows. The difference in magnitude results from the way PlotVectorField scales these black arrows. The two column vectors already specify the whole matrix (black arrows), and, therefore, the images of the canonical basis vectors (blue arrows) also specify the whole linear mapping represented by our matrix. Thus, instead of talking of two column vectors we can also talk about a linear mapping in two dimensions. The green arrows are the eigenvectors of this linear mapping. They indicate two lines on which each position vector is mapped onto a vector which has the same direction (scaled by the corresponding eigenvalue, which is negative for one of the eigenvectors). Actually, there are not only two eigenvectors but all position vectors (except the origin) on the two lines indicated by the green arrows are eigenvectors. (These two lines are called "eigenspaces".) That's my idea of this stuff. I don't know whether this makes sense, but probably it is not perfectly correct. Any comments by the experts? Hope that helps, nonetheless! Martin Kraus

**Re: Finding Solution to Set of Equations with Inequality Constraints**

**Re: QUESTION (EigenVectors)**

**Re: Characteristic Polynomials and Eigenvalues**

**QUESTION (EigenVectors)**