Positive Definite Matrices: Example Solved Problems

Positive Definite Matrices

WORKED EXAMPLES

Example 1

Determine which of the following matrices are normal.

Solution:

(a) The given matrix A is not a square matrix. Hence, it is not a normal.

(b) The given matrix B is real and symmetric. Hence it is a normal matrix.

(c) The given matrix C is a Hermitian matrix and therefore it is normal.

(d) To verify the matrix D is normal or not, we have to verify it is DD^H = D^HD.

Since DD^H = D^HD, it is normal matrix

 

Example 2

Prove that the eigenvalues of A^HA are non−negative.

Solution:

If λ is an eigenvalue of A^HA, then there must exist a non−zero eigenvector X associated with λ satisfying the equality A^HAX = λX.

 0 ≤ <AX, AX> = <A* AX, X> = <A^HAX, X> = <λX, X> = λ <X, X>.

Since X is an eigenvector, it is non−zero and <X, X> is positive. Dividing by <X, X>, it gives λ≥0.

 

Example 3

Prove that the eigen values of a Hermitian matrix are real.

Solution:

Let λ denote an eigenvalue of a Hermitian matrix A and let X denote a corresponding eigenvector. Then under the Euclidean inner product,

  λ<X,X> = <λX,X> = <AX, X> = <X, A*X>

  = < X, A^HX > = < X, λ X > =  < X, X >.

Since X is an eigenvector, it is non−zero and <X, X> is also non−zero. Dividing by <X, X> then the above relation gives λ= which implies that λ is real.

 

Example 4

Determine the adjoints of the following matrices with respect to the Euclidean inner product.

Solution:

We know that A* = A^H (Euclidean inner product)

Here c and e are self−adjoint because both are Hermitian.

 

Example 5

Determine the adjoint of A under an inner product with respect to W where 

Solution:

We know that, the adjoint exists and it is

 A* = (W^H W)^‒1 A^H(W^HW)

 

Example 6

Determine which of the following matrices are Hermitian.

Solution:

We know that the condition for Hermitian is

 A=A^H (ie) A=A^H = ()^T

 

Example 7

Use the first three tests to verify the given matrix  is positive definite.

Solution:

Test 1: Form the upper triangular matrix for the matrix A by using row operations.

Since the pivot points are all positive, the matrix is positive definite.

Test 2: Find all the minors of the matrix A.

Let the given matrix 

Since all the three principal minors are positive, the matrix is positive definite.

Test 3: A is positive definite if and only if all its eigenvalues are positive.

The characteristic equation |A−λI| = 

 (6−λ) [(6−λ) (10−λ)−4] − 2[2(10−λ)−4]−2[−4+2 (6−λ)]=0

(6−λ) [60−6λ−10λ +λ²−4]−2[20−2λ−4]−2[−4+12−2λ]=0

(6 − λ) [λ² – 16λ +56] − 2[−2λ + 16] − 2[− 2λ + 8] = 0

6λ² − 96λ +336−λ³ +16λ² − 56λ +4λ − 32 + 4λ − 16 = 0

− λ³ + 22λ² − 144λ +288=0

 λ³ − 22λ² + 144λ − 288 = 0

By using synthetic division method

The eigenvalues are λ = 4, 6, 12.

Since all the three eigenvalues are positive, the given matrix is positive definite.

 

Example 8

Use all the tests to determine whether the given matrix is positive definite or not. 

 .

Solution:

Test 1:

Since the second pivot is −45 which is negative, A is neither positive definite (or) positive semidefinite.

Test 2:

= 2[55−64] − 10[110 + 16] − 2[80 + 10]

= 2[−9]−10[126]−2[90]

= −18−1260−180

= −1458

Since P₂ and P₃ are negative, A is not positive definite.

Test 3:

To find the eigenvalues, let 

⇒       (2−λ) [(5− λ) (11− λ)−64] − 10[10(11− λ)+16] − 2[80+2(5− λ) ] = 0

 (2 − λ) [55 − 5λ − 11λ + λ²−64] − 10 [110−10λ +16] − 2[80+10−2λ] = 0

 (2−λ)[λ²−16λ−9] − 10[−10λ+126] − 2[−2λ+90] = 0

 2λ²−32λ−18− λ³ +16λ² +9λ + 100λ −1260+4λ−180 = 0

 −λ³ + 18λ² + 81λ – 1458 = 0

 λ³ – 18λ² − 81λ +1458 = 0

 (λ−9) (λ²−9λ−162) = 0

 (λ−9) (λ+9) (λ−18) = 0

  The eigenvalues are λ = −9, 9, 18.

Since all the eigenvalues are not positive, it is not positive definite.

Test 4: The diagonal elements of A must be positive. The diagonal elements 2, 5 and 11 are all positive. So no conclusion can be attained from this test.

Test 5: The elements of A having the greatest absolute value must be on the diagonal of A. From the given matrix A, the element of greatest absolute value is 11. It appears on the main diagonal. Though the absolute value is on the main diagonal, we cannot come to a conclusion about be positive definite.

Test 6: We know that a_iia_jj > |a_ij|²       (i ≠ j).

When i = 1 and j=2, we have from the given matrix A,

 a₁₁=2 and a₂₂=5 and a₁₂=10

  a₁₁a₂₂ = 2(5) = 10

 a₁₂ = 10

|a₁₂|² =|10|² = 100

  a₁₁a₂₂ > |α₁₂|²

⇒ 10 > 100

Hence A is not positive definite.

 

Example 9

Determine whether the given matrix is positive definite or not.

Solution:

Let us refer the Tests 4, 5 and 6.

Test 4: The diagonal elements are 2,−4,− 14. Since they are all not positive, it is not positive definite.

Test 5: The largest element in absolute value is ‒17. It is not on the main diagonal. Hence it is not positive definite.

Test 6: Here a₁₁=2, a₂₂=−4, a₁₂=−17

 a₁₁a₂₂ = −8 > |a₁₂|² = |‒17|² = 289

But −8 < 289

  The matrix A is not positive definite.

 

Example 10

Prove that the diagonal element of a positive definite matrix must be positive.

Solution:

If A has order n×n, define X to be an n−dimensional vector having one of its components, say the k^th equal to unity and all the other components equal to zero. For this vector 0 < <AX, X> = <AX> .  = a_kk.

 

Example 11

Let  . Find the Cholesky−Decomposition for L.

Solution:

It is given that 

The Cholesky decomposition is A = LL^H

Where L is the lower triangular matrix and L^H is the Hermition matrix of H.

 

Example 12

Find the Cholesky factorization of  

Solution:

We know that A = LL^H.

Where L is the lower triangular matrix and L^H is the Hermitian matrix of L.

Since L is the lower triangular matrix, it is in the form of

The given matrix A is in the form 

Where a₁₁=9, a₁₂=15i, a₂₁=−15i and a₂₂=74.

We start from the first column and its diagonal entry.

Let l₁₁ = √a₁₁ = √9 = 3

Then l₂₁ = a₂₁/l₁₁ = −15i / 3 = ‒5i.

Now we move to the second column and its diagonal entry

 l₂₂ = √[ a₂₂−l₂₁ ]

= √[ 74 − (−5i) (5i) ]

= √ [ 74 + 25(i²) ]

= √[ 74−25 ] = √49 = 7

To verify the Cholesky decomposition

Let us verify A = LL^H.

Where

 A = LL^H

The Cholesky−Decomposition is verified.

 

Example 13

Compute the Cholesky decomposition of .

Solution:

We know that the Cholesky decomposition is A=LL^H.

Where L is the lower triangular matrix and L^H is the Hermitian matrix of L.

We start from the first column.

Let l₁₁ = √a₁₁ = √4 = 2.

The second entry is l₂₁ = a₂₁/l₁₁ = −6i/2 = −3i

The third entry is l₃₁ = a₃₁/l₁₁ = 0/4 = 0

We now proceed to the second column, where we start with the entry

l₂₂ = √[a₂₂−l₂₁]

l₂₂=√[10−(−3i) (3i)]               ( l₂₁ = −3i and  = 3i)

= √[10 + 9i²] = √[10−9] = √1

 l₂₂ = 1

The entry below l₂₂ is l₃₂.

 l₃₂ = 1/l₂₂ (a₃₂ − l₃₁)

= 1/1 ( 6–0.(3i) ) = (6 – 0)

 l₃₂ = 6

Now we can go to the third column and find

 l₃₃=√ [ a₃₃−l₃₁−l₃₂  ]

 l₃₃ = √[ 52−0.0−6.6]

                   (l₃₁ = 0,  = 0 and l₃₂ = 6,  = 6)

l₃₃ = √[52−36] = √16 =4

Hence the triangular matrix used in the decomposition is

To verify the Cholesky decomposition

 A = LL^H Cholesky decomposition is verified.

 

Example 14

Determine the Cholesky decomposition for the given matrix

Solution:

We know that the Cholesky decomposition is A=LL^H.

 L is the lower triangular matrix and L^H is the Hermitian matrix of L.

We start from the first column

Let l₁₁=√a₁₁ = √4 = 2.

The second entry is l₂₁ = a₂₁/l₁₁ = −i2/2 = −i

The third entry is l₃₁ = a₃₁/l₁₁ = i/2

We now proceed to the second column, where we start

with the entry l₂₂ = √[a₂₂−]

= √[ 10 − (−i)(i) ]

            (l₂₁ = − i then  = i)

= √[10+ i²] = √[10 – 1] = √9

l₂₂ = 3

The entry below l₂₂ is l₃₂.

 1₃₂ = 1/2

Now we can go the third column and find

 The triangular matrix used in the decomposition is

To Verify the Cholesky Decomposition

 A=LL^H. The Cholesky decomposition is verified.

 

Example 15

Determine the Cholesky decomposition for the matrix

Solution:

We know that the Cholesky decomposition is A=L.L^H.

 Here L is the lower triangular matrix in the form

  and L^H is the Hermitian matrix of L.

The given matrix is 

We start from the first column.

Let l₁₁ = √a₁₁ = √3

The second entry is l₂₁ = a₂₁ / l₁₁ = 1/√3

The third entry is l₃₁ = a₃₁ / l₁₁ = 1/√3

Now we proceed to the second column, where we start with the entry

 l₂₂ = √ [ a₂₂ − l₂₁  ]

 = √[ 3 – 1/√3.1/√3 ]

= √[ (9 – 1)/3 ]

 l₂₂ = √(8/3)

The entry below l₂₂ is l₃₂

  l₃₂= ‒ 1/√6

Now we can go to the third column and find

 The lower triangular matrix used in the decomposition is

To verify the Cholesky decomposition

We have A = LL^H

A = LL^H. Hence the Cholesky decomposition is verified.

 

Example 16

Prove that if A = [a_ij] is an n×n positive definite matrix, then for any distinct i and j (i, j = 1, 2, 3 ... n), a_iia_ij> |aij|².

Solution:

Define X to be an n−dimensional vector having all components equal to zero except for the i^th and j^th components. Denote these as x_i and x_j respectively. For this vector

Setting x_i=−a_ij/a_ii and x_j= 1, we find that the first two terms on the right cancel and we are left with

0 < ‒a_ija_ij'/a_ii + a_jj = 1/a_ii (‒a_jia_ij + a_iia_jj)

The desired inequality follows, since a_ii is positive and because A is Hermitian a_ji = 

 

Example 17

Show that the largest element in absolute value of a positive definite matrix must lie on the main diagonal.

Solution:

Assume that the largest element in absolute value does not lie on the main diagonal but rather in another location, say the (i, j) position, with i≠j. Then |a_ij| > a_ii and | a_ij | > a_jj. It follows that |a_ij|²=|a_ij||a_ij| > a_iia_jj, which is a contradiction. Hence the assumption is incorrect.

 

Example 18

Prove that the eigenvalues of a positive definite matrix are positive.

Solution:

Let A be positive definite with eigenvalue λ and corresponding eigenvector X. Then for this X,

 0 < <AX, X> = <λX, X> = λ<X,X>

                          ……….(1)

Since X is an eigenvector, it is not zero and <X, X> is positive. Dividing (1) by <X, X> we obtain λ >0.

 

Example 19

Prove that if all the eigenvalues of a Hermitian matrix are positive, then the matrix is positive definite.

Solution:

An n×n Hermitian matrix has a canonical basis of orthonormal eigenvectors. Denote these basis vectors as X₁, X₂, X₃... X_n with corresponding eigenvalues λ₁, λ₂, λ₃ ... λ_n. Then AX_i = λ_i X_i (i = 1, 2... n).

If X is any non−zero n−dimensional vector, then the set {X₁, X_{2 ,} X₃ ..., X_n } is linearly dependent. But eigenvectors are linearly independent. It follows that there exist constants d₁, d₂, d₃... d_n such that

X= d₁X₁ + d₂X₂ + d₃X₃ + ... + d_nX_n

AX= d₁AX₁ + d₂AX₂ + ... + d_nAX_n

 = d₁λ₁X₁ + d₂λ₂X₂ + ... + d_nλ_nX_n

 <AX, X> = <d₁λ₁X₁ + d₂λ₂X₂  ... + d_nλ_nX_n, d₁X₁ + d₂X₂ + ... + d_nX_n>

= |d₁|²λ₁ +|d₂|²λ₂ + ... + |d_n|²λ_n

because the eigenvectors are orthonormal. Since the eigenvalues are given to be positive, this last quantity is positive for any non−zero vector X. Thus the matrix A is positive definite.

 

Example 20

Show that the determinant of a positive definite matrix is positive.

Solution:

The determinant of a matrix is the product of its eigenvalues, and each eigenvalue of a positive definite matrix is positive.

 

Example 21

Show that all principal minors of a positive definite matrix must be positive.

Solution:

Let A be an n×n positive definite matrix, and let B be a submatrix of A obtained by deleting from A its last k rows and k columns (k = 0, 1, 2, 3 ... k−1). Then B has order (n−k) × (n−k). Let Y denote an arbitrary non−zero (n−k) dimensional vector, and define X to be an n−dimensional vector having its first (n−k) components identical to those of Y and its last k components equal to zero, then

 0 < <AX, X> = <BY, Y>.

Since this is true for non−zero Y, it follows that B is positive definite and that |B| is positive.

 

Example 22

Show that a positive definite matrix is invertible.

Solution:

The determinant of a positive definite matrix is positive and so non−zero and therefore, that matrix must have an inverse.

Example 23

Find the square root of a matrix A = .

Solution:

The characteristic polynomial is |A−λI| = 0

 |A−λI| = 

⇒ (6−λ) [(6−λ) (10−λ)−4] − 2[2(10 − λ) − 4]

−2 [−4+2 (6−λ)] = 0

⇒ (6−λ) [60−6λ−10λ+λ²−4]−2 [20−2λ−4] −2[−4+12−2λ]=0

 (6−λ) [λ²−16λ+56] + [4λ−32] − 2 [−2λ+8] = 0

6λ² − 96λ +336−λ³ + 16λ² − 56λ + 4λ − 32 + 4λ − 16 = 0

 −λ³ +22λ² − 144λ + 288 = 0

  λ³−22λ² + 144λ−288−0

By using synthetic division method, we can find the roots.

 (λ−4) (λ²−18λ+72)=0

(λ−4) (λ−6) (λ−12)=0

 The eigenvalues are λ=4, 6, 12.

To find the eigenvectors:

To find the eigenvectors, (A−λI)X=0

(6−λ)x₁+2x₂−2x₃=0

2x₁+(6−λ)x₂−2x₃=0

−2x₁−2x₂+(10−λ)x₃=0

                 …………….(1)

Case (i)

When λ= 4, the system of equations (1) becomes

2x₁ + 2x₂ − 2x₃ = 0

2x₁+2x₂−2x₃=0

−2x₁−2x₂+6x₃ = 0

By solving the last two equations, we get

 x₁+x₂−x₃=0

 −x₁−x₂+3x₃=0

Case (ii)

When λ=6, the system of equations (1) becomes

0x₁+2x₂‒2x₃=0

2x₁+0x₂‒2x₃=0

−2x₁−2x₂+4x₃=0

Solving the last two equations

2x₁ +0x₂‒2x₃ = 0

−2x₁‒2x₂+4x₃ = 0

Case (iii)

When λ= 12, the system of equations (1) becomes

− 6x₁+2x₂−2x₃ = 0

2x₁−6x₂−2x₃ = 0

−2x₁‒2x₂−2x₃ = 0

Considering the first two equations, we get

−3x₁+x₂ − x₃ = 0

x₁−3x₂−x₃=0

Every square matrix A is similar to a matrix J in Jordan canonical form. If M is a modal matrix for A, then

  A = MJM^‒1