Linear Algebra for Machine Learning and Data Science (5)

Solving systems of linear equations

This week, you will learn how to solve a system of linear equations using the elimination method and the row echelon form. You will also learn about an important property of a matrix: the rank. The concept of the rank of a matrix is useful in computer vision for compressing images.

Learning Objectives

---

• Solve a system of linear equations using the elimination method.
• Use a matrix to represent a system of linear equations and solve it using matrix row reduction.
• Solve a system of linear equations by calculating the matrix in the row echelon form.
• Calculate the rank of a system of linear equations and use the rank to determine the number of solutions of the system.

Solving systems of linear equations: Elimination

Solving non-singular systems of linear equations

012

Ways to solve and the walkthrough of solving equations

If two equations are true, then so are the calculations

Try to solve this

a = 3, b = 8

Correct! The system of equations has a unique solution at point (3, 8), meaning that a = 3, b = 8. You can verify these values by replacing them in the equations.

Solving singular system of linear equations

It has 1 degree of freedom

The system above has 1 degree of freedom because the variable "a" can be any number and "b" is a constant minus "a"

Since it has 1 variable that can be any number it has 1 degree of freedom

No solution (Contradictory system)

Try to solve this

Infinitely many solutions

Equation 2 is (2 ⋅ Equation 1), meaning there are infinitely many solutions.

Solving a system of equations with more variables

012

Elimination method

Matrix row-reduction

In the intermediate system (aka row echelon form), we get b = 2 by using the elimination method and solving for b

The reduced row echelon form is the solved system represented as a matrix

Row echelon form for a singular system

We get the second equation as 0a + 0b = 0 since we can't solve the system

3 rules for row echelon form:

1. Diagonal numbers can be 0 or 1
2. The lower part of the diagonals is always zero
3. The upper part of the diagonals can be any number or 0 based on:
   1. If the diagonal number is 1, then it can be any number
   2. If the diagonal number is 0, then it’s 0

Row operations that preserve singularity

Rules used in linear equations can also be applied to matrices

Following manipulations preserve the singularity and non-singularity of the matrices

012

Manipulations that preserve singularity

 

All the information here is based on the Linear Algebra for Machine Learning and Data Science | Coursera from DeepLearning.AI