Sometimes people ask what math they need for machine learning. The answer depends on what you want to do, but in short our opinion is that it is good to have some familiarity with linear algebra and multivariate differentiation.
Linear algebra is a cornerstone because everything in machine learning is a vector or a matrix. Dot products, distance, matrix factorization, eigenvalues etc. come up all the time.
Differentiation matters because of gradient descent. Again, gradient descent is almost everywhere*. It found its way even into the tree domain in the form of gradient boosting - a gradient descent in function space.
We file probability under statistics and that’s why we don’t mention it here.
Now for some recommendations. Trey Causey once tweeted about Gilbert Strang’s linear algebra course. We like it very much.
- a youtube playlist
- the book: Introduction to linear algebra
- course page at MIT OCW
Executing a determinants formula in lecture 19
If you like books, there are a few free books online, for example Linear algebra by Jim Hefferon. Also, people put up PDFs of pretty much any book you can imagine, you can google them if you lost your copy.
For shorter intro, check out an upcoming book about deep learning by Yoshua Bengio, Ian Goodfellow and Aaron Courville. It has three separate chapters on linear algebra, probability and numerical computation. The drafts are online.
MOOCs
There are a few MOOCs about linear algebra specifically. They differ in style.
- Coding the Matrix: Linear Algebra through Computer Science Applications by Philip Klein, Brown University.
- Linear Algebra - Foundations to Frontiers by Robert van de Geijn, University of Texas.
- Applications of Linear Algebra, Part 1 and Part 2. A newer course by Tim Chartier, Davidson College.
A shorter MOOC at Coursera, with linear algebra and some calculus for financial applications. The lecturer looks convincing:
- Mathematical Methods for Quantitative Finance by Kjell Konis, University of Washington.
And here’s a calculus course with some linear algebra:
- Massively Multivariable Open Online Calculus Course from the Ohio State University - the course is a first taste of multivariable calculus, but viewed through the lens of linear algebra.
More math MOOCs
- Convex Optimization - a MOOC on optimization from Stanford, by Steven Boyd, an authority on the subject.
- How to Learn Math: For Students - deals with basic attitude towards math.
- Street-Fighting Math - “shoot first, ask questions later” approach. It teaches how to guess answers without a proof or an exact calculation, in order to develop insight.
- Introduction to Mathematical Thinking from Stanford.
*Almost in a mathematical sense, that is everywhere except a finite number of places.