I want to study Deep Learning. What subjects should I learn to ensure I’m prepared for the material in a typical Deep Learning textbook?
The book “Deep Learning,” by Ian Goodfellow, Yoshua Bengio and Aaron Courville (three eminent researchers in the field), recommends familiarity the following subjects:
- Linear Algebra
- Multivariable Calculus
- Probability
- Information Theory
- Numerical Methods
- Machine Learning
Studying these subjects at length is a bit overkill. Stanford’s famous course, C231n: Convolutional Neural Networks for Visual Recognition, assumes knowledge of a small subset of these subjects:
- Basic matrix algebra (addition, multiplication)
- Derivatives of multivariable functions
- Basic probability theory (random variables, expected value, variance)
If you’d like to study these prerequisites in-depth, there are amazing online resources.
- MIT’s Linear Algebra course
- MIT’s Single Variable Calculus course
- MIT’s Multivariable Calculus course
- MIT’s Probability course
- Stanford’s Machine Learning course material and online lectures.
Once you’re ready to study deep learning…
Some online courses:
- Stanford’s CS231n course material and online lectures
- Deep Learning Specialization. A series of courses focused on Deep Learning.
Textbooks:
- Neural Networks and Deep Learning, by Michael Nielson is a great introduction
- deeplearningbook.org, by Goodfellow et. al, is a more in-depth exploration
Online resources:
- deeplearning.net – All things deep learning
2 Likes
Bill Gosper – legenday hacker who for many years kept recapturing the record for calculating PI to the most millions of digits, back in the days when you carried your 640K of memory around on a disc-pak weighing five pounds – was once asked the best book for an introduction to computation. “Alice in Wonderland.”
“Why?”
“Because it’s the best book to introductory everything.”
I guess that would go for deep learning, too, woonit?
1 Like