Linear algebra is all about matrices, and mostly about multiplying those matrices. If you were to remember only one thing about the topic, it would be this. To multiply two matrices, they have to be of certain shape. Specifically, the number of columns in the first matrix must be the same as the number of rows in the second matrix. That’s because we’re multiplying rows by columns, and they have to be of the same length.

For example (the notation is made up):

(1000, 4) * (4,50) => (1000, 50)

Note how the resulting matrix has the same number of rows as the first one and the same number of columns as the second one. This knowledge of shape requirements is very helpful in debugging.

Linear algebra resources

If you would like to learn more about linear algebra, there are many courses and books about the topic, many of them free. For beginners, Jeff Chasnov’s Matrix Algebra for Engineers seems like a good choice. The videos are available on YouTube, and the booklet is linked in the descriptions. For starters, you can check the video about matrix multiplication.

The course is good, although in our opinion it contains some irrelevant parts. For example, it’s quite difficult to imagine that in practice you would ever need to compute a matrix determinant using the the Leibniz formula. What is that? You simply don’t need to know, unless you are a mathematician. Don’t clutter your memory.

Jeff Chasnov offering you a blue pill and a red pill

For a deeper dive Gilbert Strang’s lectures are a classic and a gold standard. Strang also has a few books.

For doing exercises, one could use sympy in addition or even instead of pen and paper. Sympy seems to be more geared towards educational use than numpy and features operations you won’t find in numpy, such as computing matrix determinants, for example.

In [1]: from sympy import *

In [2]: A = Matrix([ [1,2,3], [4,5,6], [7,8,9] ])

In [3]: A
Out[3]:
Matrix([
[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])

In [4]: A.det()
Out[4]: 0

There is even a more advanced linear algebra course that uses sympy with code snippets embedded on the webpage in a notebook fashion.

Credit: A meme page to check every time MatLab crashes

If interested, check our ten year old article on math for machine learning listing some more material, particularly online courses. Unfortunately many of the links have expired, but not all of them.