Linear Algebra for Machine Learning and Data Science (17)

Determinants and Eigenvectors

Check your knowledge

Q1

If $\text{det } M = 20$ and $\text{det } N = 10$, and $M, N$ have the same size. What is the value of $\text{det } M\cdot N$ and $\text{det}(N^{-1})$? (Videos: Determinant of a product, Determinant of inverses)

Answer

$\det M\cdot N$ is 200 and $\det(N^{-1})$ is 1/10 or 0.1

Q2

Does the set $\{(1,2),(4,8)\}$ form a base for $\mathbb{R}^2$? (Videos: Bases in Linear Algebra, Span in Linear Algebra)

Answer

No, two points form a linear line (not linearly independent) and can’t cover the 2-dimensional space.

Q3

Compute the characteristic polynomial of the following matrix. Use it to find its eigenvalues and eigenvectors. (Videos: Eigenvalues and eigenvectors)

$$ M=\left[\begin{array}{cc}1&2\\3&5\end{array}\right] $$

Answer

The characteristic polynomial is $(1-\lambda)(5-\lambda)-6=0$

Solving for lambda, the eigenvalues are $\pm\sqrt{10}+3$

Eigenvectors are $[-1.72, 1]$ and $[0.39, 1]$

Q4

Does there exist an eigenbasis for the linear transformation defined by $M$ above? (Video: Eigenbases)

Answer

It has two linearly independent eigenvalues and eigenvectors, so an eigenbasis exists.

Refer to this, this, and this for more information.

Q&A

Question 1: Can you explain the characteristic polynomial of a matrix?

A characteristic polynomial is a way to find the eigenvalues of the given matrix. First, eigenvalues can be found by subtracting variable lambda times the identity matrix from the given matrix. Then we set the determinants of the resulting matrix to 0 and solve for lambda. The solved values are the eigenvalues of the given matrix.

Question 2: How would you describe the span of vectors?

The span of vectors refers to the set of all possible linear combinations of those vectors. It represents the subspace that can be reached by scaling and adding those vectors together. The span is not limited to a plane and can be a line, a plane, or even the entire space depending on the number of linearly independent vectors.

Question 3: How would you compute the covariance matrix for Principal Component Analysis (PCA)?

Computing the covariance matrix follows these steps:

1. Get two matrices from the original matrix: The transposed matrix and the original matrix
2. Get the mean of each column (feature) and subtract it from each value in both matrices
3. Multiply the transposed matrix with the original matrix to obtain the covariance matrix

Question 4: How would you explain the concept of the rank of a linear transformation?

The rank of a linear transformation refers to the maximum number of linearly independent vectors in the output space of the transformation. It represents the dimension of the vector space spanned by the transformed vectors. Additionally, the rank of a linear transformation provides information about the dimension of the image or range of the transformation.

Question 5: Can you describe the properties of square matrices?

• Singularity: Square matrices can be classified as singular or non-singular based on their determinants
• Determinants: The determinant of a square matrix is a scalar value that provides information about the matrix's properties, such as invertibility and volume scaling
• Symmetry: Some square matrices exhibit symmetry, where the elements above and below the main diagonal are reflections of each other
• Skew-symmetry: Skew-symmetric matrices have elements that are negations of their corresponding elements across the main diagonal
• Diagonalizability: Square matrices can be diagonalizable, meaning they can be transformed into a diagonal matrix using a similarity transformation
• Invertibility: Non-singular square matrices are invertible, meaning they have an inverse matrix that when multiplied together, yields the identity matrix
• Orthogonality: Orthogonal matrices are square matrices whose columns and rows are orthogonal unit vectors

Question 6: How would you explain the concept of the determinant of a matrix product?

The determinant of a matrix product is equal to the product of the determinants of the individual matrices involved in the multiplication. In other words, if you have two matrices A and B, the determinant of their product AB is equal to the determinant of A multiplied by the determinant of B. This property allows us to compute the determinant of a matrix product by simply multiplying the determinants of the individual matrices involved. It provides a useful tool for evaluating the determinant of a larger matrix by breaking it down into smaller matrix products.

Question 7: What is a determinant

The determinant is a scalar value associated with a square matrix. It is a mathematical property that provides important information about the matrix. The determinant can be thought of as a measure of how the matrix transforms space.

Geometrically, the determinant represents the scaling factor of the volume or area of a parallelepiped (a higher-dimensional analog of a parallelogram or parallelepiped) after the matrix transformation is applied. It tells us how much the matrix stretches or compresses space in different directions.

Algebraically, the determinant is calculated using a specific formula that involves the elements of the matrix. It is a combination of additions and subtractions of products of matrix elements. The determinant can be used to determine if a matrix is invertible (non-singular) or not (singular). If the determinant is zero, the matrix is singular and does not have an inverse.

 

All the information here is based on the Linear Algebra for Machine Learning and Data Science | Coursera from DeepLearning.AI