Linear Algebra for Machine Learning and Data Science (2)

Systems of linear equations

Systems of Equations

A geometric notion of singularity

01

Linear equation with and without constants

By looking at the above plots we can conclude that the constants don’t matter when it comes to determining singular and nonsingular, hence we can leave them at 0

A non-singular equation stays non-singular and a singular equation stays singular regardless of constants

Singular vs non-singular matrices

Coefficients from the linear equations are taken as the elements of the matrices

012

Linear equations to matrices and their singularities

And of course, the constants don’t matter

Linear dependence and independence

Rows are linearly dependent if any rows can be derived from another row(s)

Rows are linearly independent if rows are not derived from other ones

012

Linearly independent vs dependent and their examples

Try to solve this quiz

• Matrix 1 - Dependent
  • Correct! In matrices with linearly dependent rows, one row can be obtained from the others.
• Matrix 2 - Dependent
  • Correct! The matrix is singular, therefore the rows are linearly dependent.
• Matrix 3 - Independent
• Matrix 4 - Dependent

The determinant

Quick formula to find out whether a given matrix is singular or not based on the determinant

If the determinant is 0 then the matrix is singular otherwise non-singular

Computing determinant of the 2x2 matrix

Solve this quiz

• Matrix 1 - 16
  • Correct! The determinant is 16 by applying the formula that you saw in the video [ad-bc]. In this matrix, $\text{det}=[(5⋅3)−(−1⋅1)]=15+1=16$
  • The matrix is non-singular
• Matrix 2 - 0
  • Correct! The determinant is 0 by applying the formula that you saw in the video [ad-bc]. In this matrix, $\text{det} = [(2∗3)−(−6∗−1)]=0$.
  • The matrix is singular

Computing determinant of the 3x3 matrix

Solve this

• Matrix 1 - 0
  • Correct!
  • Computing the product of the diagonals from left to right we obtain:
  • $1⋅1⋅3+0⋅0⋅3+1⋅3⋅0=3$
  • Now, computing the product of the diagonals from right to left:
  • $1⋅1⋅3+0⋅0⋅3+1⋅3⋅0=3$
  • Therefore, $\text{det} \,\,M_1=3−3=0$.
  • Indeed, after calculating the determinant that is equal to 0, you conclude that the matrix is singular.
• Matrix 2 - 0
  • Correct!
  • Singular matrices have det = 0.
  • diagonals from left to right
  • $1 \cdot 1 \cdot -1 + 1\cdot 2\cdot 0 + 1\cdot 1 \cdot 0 = -1$
  • diagonals from right to left
  • $1 \cdot 1 \cdot 0 + 1 \cdot 1 \cdot -1 + 1 \cdot 2 \cdot 0 = -1$
  • Therefore, $-1 - (-1) = 0$
• Matrix 3 - 6
  • Correct! Non-singular matrices have determinants different than 0.
  • diagonals from left to right
  • $1 \cdot 2 \cdot 3 + 1\cdot 2\cdot 0 + 1\cdot 0 \cdot 0 = 6$
  • diagonals from right to left
  • $1 \cdot 2 \cdot 0 + 1 \cdot 0 \cdot 3 + 1 \cdot 2 \cdot 0 = 0$
  • Therefore, $6 - 0 = 6$
• Matrix 4 - 0
  • Correct! Singular matrices have determinants equal to 0.
  • diagonals from left to right
  • $1 \cdot 3 \cdot 10 + 2\cdot -2\cdot 2 + 5\cdot 0 \cdot 4 = 22$
  • diagonals from right to left
  • $5 \cdot 3 \cdot 2 + 2 \cdot 0 \cdot 10 + 1 \cdot -2 \cdot 4 = 22$
  • Therefore, $22 -22 = 0$

All the information here is based on the Linear Algebra for Machine Learning and Data Science | Coursera from DeepLearning.AI