v1 · padrão canônico

Lesson 116 — Special matrices: symmetric, orthogonal, and Hermitian

The three families of matrices that dominate applications: symmetric (A = A^T), orthogonal (Q^T Q = I), Hermitian (A^* = A). Spectral theorem, quadratic forms, and Cholesky decomposition.

Used in: 3rd year HS advanced · Equiv. German Leistungskurs (Vectors + Linear Mappings) · Equiv. Singapore H2 Math (chapter Matrices advanced)

A = A^T \Rightarrow A = QDQ^T, \quad Q^TQ = I

Choose your door

Rigorous notation, full derivation, hypotheses

Rigorous definition

Symmetric matrices

"A matrix is called symmetric if it equals its own transpose: $A = A^T$ . [...] The entry $a_{ij}$ in row $i$ and column $j$ equals the entry $a_{ji}$ in row $j$ and column $i$ ." — Beezer, A First Course in Linear Algebra, §OD

A = QDQ^T = \sum_{i=1}^{n} \lambda_i \, q_i q_i^T

what this means · Orthogonal diagonalization of symmetric A: Q is the matrix whose columns are orthonormal eigenvectors; D contains the corresponding eigenvalues on the diagonal.

The last equality shows $A$ as a sum of rank-1 projections weighted by eigenvalues — the spectral decomposition.

Orthogonal matrices

"An $n \times n$ matrix $Q$ is called an orthogonal matrix if $Q^T Q = I$ ." — Austin, Understanding Linear Algebra, §7.1

Fundamental properties:

Preserves norm: $\lVert Qv \rVert = \lVert v \rVert$ for all $v$ .
Preserves inner product: $\langle Qu, Qv \rangle = \langle u, v \rangle$ .
$\det Q = \pm 1$ . If $\det Q = 1$ : rotation. If $\det Q = -1$ : rotation followed by reflection.
Complex eigenvalues have modulus 1.

The complex analog is the unitary matrix: $U^* U = I$ , where $U^*$ is the adjoint (conjugate transpose).

Hermitian matrices

"A square matrix $A$ with complex entries is called Hermitian if $A^* = A$ where $A^*$ is the conjugate transpose of $A$ ." — Beezer, A First Course in Linear Algebra, §HMD

Complex spectral theorem. Every Hermitian matrix has real eigenvalues and admits unitary diagonalization: $A = U D U^*$ .

Positive definite matrices

Equivalent criteria for SPD:

All eigenvalues $\lambda_i > 0$ .
All leading principal minors (determinants of the $k \times k$ submatrices from the upper left corner) are positive — Sylvester's criterion.
There exists $L$ lower triangular invertible with $A = LL^T$ — Cholesky factorization.

Containment hierarchy: SPD with Cholesky ⊂ SPD ⊂ PSD ⊂ Symmetric.

Quadratic form

Classification: if all $\lambda_i > 0$ : positive definite. If all $\lambda_i < 0$ : negative definite. If there are positive and negative eigenvalues: indefinite.

Worked examples

Example— 1· Verify symmetry and classify (basic level)

Problem. Given $A = \begin{pmatrix} 3 & -1 \\ -1 & 3 \end{pmatrix}$ , verify that $A$ is symmetric and determine if it is SPD. If it is SPD, find the Cholesky factorization.

Strategy. Verify $A = A^T$ by inspection. Check SPD via Sylvester criterion. For Cholesky, solve $A = LL^T$ with $L$ lower triangular.

Solution.

Symmetry: $A^T = \begin{pmatrix} 3 & -1 \\ -1 & 3 \end{pmatrix} = A$ . Confirmed.

Sylvester: $a_{11} = 3 > 0$ ; $\det A = 9 - 1 = 8 > 0$ . So $A$ is SPD.

Cholesky: write $L = \begin{pmatrix} \ell_{11} & 0 \\ \ell_{21} & \ell_{22} \end{pmatrix}$ . Then $LL^T = \begin{pmatrix} \ell_{11}^2 & \ell_{11}\ell_{21} \\ \ell_{11}\ell_{21} & \ell_{21}^2 + \ell_{22}^2 \end{pmatrix}$ .

Comparing with $A$ : $\ell_{11} = \sqrt{3}$ ; $\ell_{21} = -1/\sqrt{3}$ ; $\ell_{22} = \sqrt{3 - 1/3} = \sqrt{8/3}$ .

$L = \begin{pmatrix} \sqrt{3} & 0 \\ -1/\sqrt{3} & \sqrt{8/3} \end{pmatrix}$

Verification. $LL^T = \begin{pmatrix} 3 & -1 \\ -1 & 1/3 + 8/3 \end{pmatrix} = \begin{pmatrix} 3 & -1 \\ -1 & 3 \end{pmatrix} = A$ . Correct.

Source. Beezer, A First Course in Linear Algebra, §PSD, p. 447 — GNU FDL.

Example— 2· Orthogonally diagonalize a 2x2 matrix (intermediate)

Problem. Orthogonally diagonalize $A = \begin{pmatrix} 5 & 4 \\ 4 & 5 \end{pmatrix}$ .

Strategy. Calculate eigenvalues, then orthonormal eigenvectors, assemble $Q$ and verify $QDQ^T = A$ .

Solution.

Characteristic polynomial: $\det(A - \lambda I) = (\lambda - 5)^2 - 16 = \lambda^2 - 10\lambda + 9 = (\lambda - 1)(\lambda - 9) = 0$ .

Eigenvalues: $\lambda_1 = 1$ , $\lambda_2 = 9$ .

For $\lambda_1 = 1$ : $(A - I)v = 0$ gives $4v_1 + 4v_2 = 0$ , so $v = \frac{1}{\sqrt{2}}(1, -1)^T$ .

For $\lambda_2 = 9$ : $(A - 9I)v = 0$ gives $-4v_1 + 4v_2 = 0$ , so $v = \frac{1}{\sqrt{2}}(1, 1)^T$ .

Since $\lambda_1 \neq \lambda_2$ , the eigenvectors are orthogonal (spectral theorem).

$Q = \frac{1}{\sqrt{2}} \begin{pmatrix} 1 & 1 \\ -1 & 1 \end{pmatrix}, \quad D = \begin{pmatrix} 1 & 0 \\ 0 & 9 \end{pmatrix}$

Verification. $QDQ^T = \frac{1}{2}\begin{pmatrix} 1 & 1 \\ -1 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & 9 \end{pmatrix} \begin{pmatrix} 1 & -1 \\ 1 & 1 \end{pmatrix} = \frac{1}{2}\begin{pmatrix} 10 & 8 \\ 8 & 10 \end{pmatrix} = \begin{pmatrix} 5 & 4 \\ 4 & 5 \end{pmatrix} = A$ .

Source. Austin, Understanding Linear Algebra, §7.2, Example 7.2.3 — CC-BY-SA.

Example— 3· Verify and use orthogonal matrix (intermediate)

Problem. Let $Q = \frac{1}{\sqrt{2}} \begin{pmatrix} 1 & -1 \\ 1 & 1 \end{pmatrix}$ . Verify that $Q$ is orthogonal, compute $\det Q$ , and interpret geometrically.

Strategy. Verify $Q^T Q = I$ , compute determinant, identify the transformation.

Solution.

$Q^T = \frac{1}{\sqrt{2}} \begin{pmatrix} 1 & 1 \\ -1 & 1 \end{pmatrix}$ .

$Q^T Q = \frac{1}{2} \begin{pmatrix} 1 & 1 \\ -1 & 1 \end{pmatrix} \begin{pmatrix} 1 & -1 \\ 1 & 1 \end{pmatrix} = \frac{1}{2} \begin{pmatrix} 2 & 0 \\ 0 & 2 \end{pmatrix} = I_2$ .

So $Q$ is orthogonal. Also: $QQ^T = I$ (the reciprocal verification follows similarly).

$\det Q = \frac{1}{2}(1 \cdot 1 - (-1) \cdot 1) = \frac{1}{2} \cdot 2 = 1$ .

Since $\det Q = 1$ , the transformation is a rotation of $45°$ counterclockwise ( $\cos 45° = \sin 45° = 1/\sqrt{2}$ ).

Lengths preserved: $\lVert Q v \rVert^2 = v^T Q^T Q v = v^T I v = \lVert v \rVert^2$ .

Verification. Apply $Q$ to the vector $e_1 = (1, 0)^T$ : $Qe_1 = \frac{1}{\sqrt{2}}(1, 1)^T$ , which has norm 1 and angle $45°$ . Consistent with a $45°$ rotation.

Source. Hefferon, Linear Algebra, chapter 5, §2, Exercise 2.15 — CC-BY-SA.

Example— 4· Classify quadratic form and identify conic (advanced)

Problem. Classify the quadratic form $Q(x, y) = 5x^2 + 6xy + 5y^2$ and identify the curve $Q(x, y) = 8$ .

Strategy. Write matrix form, orthogonally diagonalize, obtain canonical form, identify the conic by eigenvalues.

Solution.

Associated matrix: $A = \begin{pmatrix} 5 & 3 \\ 3 & 5 \end{pmatrix}$ (note: the coefficient of the cross term $6xy$ is divided by 2 for off-diagonal elements, since $x^T A x = 5x^2 + 2 \cdot 3 \cdot xy + 5y^2$ ).

Eigenvalues: $\det(A - \lambda I) = (5-\lambda)^2 - 9 = 0 \Rightarrow \lambda_{1,2} = 5 \mp 3$ , so $\lambda_1 = 2$ , $\lambda_2 = 8$ .

Both positive $\Rightarrow$ $A$ is SPD $\Rightarrow$ $Q(x, y)$ is positive definite.

In the eigenvector basis (45° rotation, $u = (x+y)/\sqrt{2}$ , $v = (y-x)/\sqrt{2}$ ):

$Q = 2u^2 + 8v^2$

The curve $2u^2 + 8v^2 = 8$ is equivalent to $u^2/4 + v^2/1 = 1$ . This is an ellipse with semi-major axis $a = 2$ (along $u$ ) and semi-minor axis $b = 1$ (along $v$ ).

Verification. The ellipse axes are the eigenvector directions: $u$ -direction is $(1/\sqrt{2}, 1/\sqrt{2})$ (eigenvector of $\lambda_1 = 2$ ) and $v$ -direction is $(1/\sqrt{2}, -1/\sqrt{2})$ (eigenvector of $\lambda_2 = 8$ ). At the extreme of the major axis with $u = 2, v = 0$ : the corresponding $(x, y)$ is $(\sqrt{2}, \sqrt{2})$ . Then $Q(\sqrt{2}, \sqrt{2}) = 2 \cdot 4 + 0 = 8$ . Verified.

Source. Austin, Understanding Linear Algebra, §7.3, Example 7.3.4 — CC-BY-SA.

Example— 5· Demonstrate that A^T A is always PSD (proof level)

Problem. Show that $A^T A$ is always symmetric and positive semidefinite for any $A \in \mathcal{M}_{m \times n}(\mathbb{R})$ .

Strategy. Check symmetry by direct transposition. Check PSD by computing $x^T (A^T A) x$ and recognizing it as a squared norm.

Solution.

Symmetry. $(A^T A)^T = A^T (A^T)^T = A^T A$ . So $A^T A$ is symmetric.

PSD. For any $x \in \mathbb{R}^n$ :

$x^T (A^T A) x = (Ax)^T (Ax) = \lVert Ax \rVert^2 \geq 0$

So $A^T A$ is positive semidefinite.

When is it SPD? When $\lVert Ax \rVert = 0 \Rightarrow x = 0$ , that is, when $A$ has full rank ( $n$ linearly independent columns).

Consequence: the Gram matrix $G_{ij} = v_i \cdot v_j = V^T V$ (where $V$ has the $v_i$ as columns) is always PSD, and SPD if and only if the vectors $v_1, \ldots, v_n$ are linearly independent.

Verification. For $A = \begin{pmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 1 \end{pmatrix}$ : $A^T A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$ , which has eigenvalues $1$ and $3$ , both positive. SPD (full rank).

Source. Beezer, A First Course in Linear Algebra, §PSD, Theorem PSMT — GNU FDL.

Exercise list

42 exercises · 10 with worked solution (25%)

Application 25Understanding 5Modeling 6Challenge 3Proof 3

Ex. 116.1ApplicationAnswer key
Verify that is symmetric.
Solve online
Ex. 116.2ApplicationAnswer key
Verify via Sylvester criterion that is SPD.
Solve online
Ex. 116.3Application
Verify that is orthogonal by computing .
Solve online
Ex. 116.4ApplicationAnswer key
Show that the planar rotation matrix has determinant 1 for any .
Solve online
Ex. 116.5Application
Compute the Cholesky factorization of .
Solve online
Ex. 116.6Application
Compute the eigenvalues of .
Solve online
Ex. 116.7Application
Find orthonormal eigenvectors of corresponding to and .
Solve online
Ex. 116.8Application
Assemble the orthogonal diagonalization of using the eigenvectors computed in the previous exercise.
Solve online
Ex. 116.9Application
Verify whether is SPD or only PSD.
Solve online
Ex. 116.10Application
Write the matrix of reflection with respect to the $x$ axis in $\mathbb{R}^2$ . Verify that it is orthogonal and compute its determinant.
Solve online
Ex. 116.11Application
Are the columns of the matrix orthonormal? Is the matrix orthogonal? Justify.
Solve online
Ex. 116.12Application
Determine the eigenvalues of and classify: SPD, PSD, or indefinite.
Solve online
Ex. 116.13Application
Compute the Cholesky factorization of .
Solve online
Ex. 116.14Application
Write the symmetric matrix associated with the quadratic form .
Solve online
Ex. 116.15Application
Classify the quadratic form using the eigenvalues of the associated matrix.
Solve online
Ex. 116.16Application
Classify the quadratic form and identify the curve .
Solve online
Ex. 116.17Application
Identify the type of conic and find the semi-axes for the curve .
Solve online
Ex. 116.18ApplicationAnswer key
Classify the quadratic form via eigenvalues.
Solve online
Ex. 116.19Application
Show that the product of two orthogonal matrices and is orthogonal.
Solve online
Ex. 116.20ApplicationAnswer key
Write the $3 \times 3$ permutation matrix that swaps rows 1 and 2. Verify that it is orthogonal and compute its determinant.
Solve online
Ex. 116.21Application
Compute for using orthogonal diagonalization.
Solve online
Ex. 116.22ApplicationAnswer key
Verify for a $2 \times 2$ symmetric matrix that (Frobenius norm squared).
Solve online
Ex. 116.23UnderstandingAnswer key
Is every matrix with real positive eigenvalues positive definite?
Solve online
Ex. 116.24UnderstandingAnswer key
What can be concluded about the (complex) eigenvalues of an orthogonal matrix?
Solve online
Ex. 116.25Understanding
When are eigenvectors of a real symmetric matrix guaranteed to be orthogonal?
Solve online
Ex. 116.26Application
Let with (Householder reflection). Verify that is orthogonal and symmetric.
Solve online
Ex. 116.27Application
Show that any square matrix can be written as where is symmetric and is skew-symmetric (that is, ).
Solve online
Ex. 116.28Application
Classify the quadratic form using Sylvester criterion and identify the curve .
Solve online
Ex. 116.29Modeling
Explain why the sample covariance matrix of variables with observations is always PSD, and under what condition is it SPD.
Solve online
Ex. 116.30Modeling
In classical mechanics, the inertia tensor of a rigid body is a $3 \times 3$ matrix. Explain why it is symmetric and what its eigenvectors and eigenvalues represent physically.
Solve online
Ex. 116.31ModelingAnswer key
At a critical point of , the Hessian is symmetric. Explain why SPD implies that is a strict local minimum.
Solve online
Ex. 116.32ModelingAnswer key
In financial Monte Carlo simulation, explain how Cholesky factorization of an SPD covariance matrix is used to generate correlated random vectors.
Solve online
Ex. 116.33Modeling
In image compression (JPEG), the DCT matrix is orthogonal. Explain how this property ensures that compressing (discarding some coefficients) causes controlled information loss.
Solve online
Ex. 116.34Modeling
The Laplacian of an undirected graph with positive weights is symmetric and PSD. Show why .
Solve online
Ex. 116.35Understanding
Why does every real symmetric matrix have real eigenvalues? (Prove using the complex inner product.)
Solve online
Ex. 116.36Understanding
Prove that eigenvectors of a symmetric matrix corresponding to distinct eigenvalues are orthogonal.
Ex. 116.37Challenge
Prove that every orthogonal matrix has determinant .
Ex. 116.38Challenge
If is orthogonal and (complex eigenvalue), show that .
Solve online
Ex. 116.39Challenge
Prove that is orthogonal if and only if preserves all inner products: for all .
Ex. 116.40Proof
Prove the spectral theorem for $2 \times 2$ symmetric matrices: every is orthogonally diagonalizable.
Ex. 116.41Proof
Prove that a symmetric matrix is SPD if and only if there exists invertible with .
Ex. 116.42Proof
Prove that and have the same nonzero eigenvalues (with the same multiplicity).

Sources

Beezer, A First Course in Linear Algebra — Rob Beezer · 2022 · EN · GNU FDL · §OD (Orthonormal Diagonalization), §PSD, §HMD. Primary source for this lesson.
Hefferon, Linear Algebra — Jim Hefferon · 2020 · EN · CC-BY-SA · chapter 5 (Similarity) and chapter 5, §III (Bilinear forms).
Austin, Understanding Linear Algebra — David Austin · 2023 · EN · CC-BY-SA · §7.1–7.3 (Orthogonal Diagonalization, Quadratic Forms).