Linear Algebra for Machine Learning and Data Science (7)

Solving systems of linear equations

Solving system of linear equations: Row echelon form and rank

The rank of a matrix

Image with different ranks

Rank here is referred to as the amount of information/data it holds

012

Singular vs non-singular with rank numbers

Try to solve this

Matrix 1

2

As you may have noticed, the matrix is non-singular. The rank of the matrix is 2, as you obtain 2 new pieces of information.

Matrix 2

1

The rank is 1. You can find the rank using different methods from the video. The rank is the largest number of linearly independent rows/columns in the matrix.

The rank of a matrix in general

There's an easier way to find the rank of the matrix and it has to do with the row echelon form of the matrix

Row echelon form

We can think of the row echelon form as solving for a variable

0123

Getting the row echelon form for the (non)singular matrix and the rank info

Row echelon form in general

012345

As mentioned before, row echelon form is like the intermediate step of solving a system of equations by eliminating the leftmost variable and isolating the rightmost variable, then turning the coefficients into a matrix

For a singular matrix, we update the rows to 0 for the variables that we don't have unique solutions (can't solve)

Reduced row echelon form

01234

All about reduced row echelon form

Reduced row echelon form is the solved system's coefficients turned into a matrix

The Gaussian Elimination Algorithm

Turn the matrix into a row echelon form and also apply operations to the column of constants (last column)

012345678

The elimination method applied to constants as well

Then apply back substitution to the reduced echelon form

012

Back substitution

 

All the information here is based on the Linear Algebra for Machine Learning and Data Science | Coursera from DeepLearning.AI