Suggested reading: getting going with deep learning

Based on a conversation we had in the Machine Listening Lab last week, here are some blogs and other things you can read when you’re – say – a new PhD student who wants to get started with applying/understanding deep learning. We can recommend plenty of textbooks too, but here it’s mainly blogs and other informal introductions. Our recommended reading:

Andrew Ng’s coursera course on “Deep Learning”
– it’s not free to be a student on the course, BUT it is free to “audit” the course, either by signing in, or simply by watching the videos on Youtube

A brief overview of Deep Learning – a v good intro. (Also: DO read the comments. Some big names give their thoughts.)
http://yyue.blogspot.co.uk/2015/01/a-brief-overview-of-deep-learning.html

This overview Nature paper is good too:
http://www.nature.com/nature/journal/v521/n7553/full/nature14539.html

This blog post series covers the LINEAR ALGEBRA underlying deep learning and numerical optimisation
https://hadrienj.github.io/posts/Deep-Learning-Book-Series-Introduction/

Introductions that show all the different types of NN architectures:
https://towardsdatascience.com/the-mostly-complete-chart-of-neural-networks-explained-3fb6f2367464
https://blog.statsbot.co/neural-networks-for-beginners-d99f2235efca

Deep learning book (free online)  by Ian Goodfellow and Yoshua Bengio and Aaron Courville
https://www.deeplearningbook.org/

Deep learning book (free online) by Michael Nielsen
http://neuralnetworksanddeeplearning.com/

PRACTICAL:

My Neural Network isn’t working! What should I do?
http://theorangeduck.com/page/neural-network-not-working
…you’ll need this!

ADVANCED:

a very readable tutorial on image generation using deep learning (specifically, GANs)
http://bamos.github.io/2016/08/09/deep-completion/

For Multi Task Learning:
http://ruder.io/multi-task/index.html

For LSTMs (a popular type of recurrent neural network):
http://colah.github.io/posts/2015-08-Understanding-LSTMs/