Deep Learning Goodfellow et al: The Book by the Masters

Review

The following is a review of the book Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville.

Deep Learning by Ian Goodfellow et al is the reference, go to book in universities for teaching the theory behind Deep Learning. It is an exhaustively written book, with lots of theory and maths, oriented towards granting its readers a deep understanding of what happens in the guts of a Deep Artificial Neural Network. Because of this goal, the book contains very little code or programming references. It is no surprise then, that this book was written by 3 of the top personalities in the world of Deep Learning: Ian Goodfellow, Yoshua Bengio (the Godfather of Deep Learning) and Aaron Courville.

We would recommend this book for those familiar with Machine Learning implementations that want to get a profound grasp of all the theory behind the code that they are familiar with, as its topics overflow the pure Deep Learning to cover topics like Linear Algebra, Calculus, Signal Processing, etc…and then carries on to teach about how all those concepts are used in different neural network architectures like Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short Term Memory Networks (LSTMs) or Generative Adversarial Netowkrs (GANs).

Despite having its focus on covering the mathematical and conceptual background behind Deep Learning, Goodfellow and the others also showcase deep learning techniques used in Industry, and the research perspectives of the field. You will learn everything from basic fast-forward Neural Network structures to regularisation and optimisation algorithms, and the most advanced ANN architectures and autoencoders.

Deep Learning by Goodfellow et al can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms.

More Information about this book can be found on the following link http://www.deeplearningbook.org/

Book description and contents

Deep Learning is an exhaustive and heavy book, aimed to make its readers experts in almost every subject surrounding an Artificial Neural Network. It is not a book for the faint of heart, so those who prefer a lighter or more practical approach to learning should probably look for another text like Deep Learning from Scratch by Seth Weidman. Having said this, the current book by Goodfellow, Bengio and Courville, if internalised will make you able to understand, fully comprehend, and debug neural networks with the highest possible confidence.

The contents of the book are the following:

• Introduction
• Part I: Applied Math and Machine Learning Basics
  • 2 Linear Algebra
  • 3 Probability and Information Theory
  • 4 Numerical Computation
  • 5 Machine Learning Basics
• Part II: Modern Practical Deep Networks
  • 6 Deep Feedforward Networks
  • 7 Regularization for Deep Learning
  • 8 Optimization for Training Deep Models
  • 9 Convolutional Networks
  • 10 Sequence Modeling: Recurrent and Recursive Nets
  • 11 Practical Methodology
  • 12 Applications
• Part III: Deep Learning Research
  • 13 Linear Factor Models
  • 14 Autoencoders
  • 15 Representation Learning
  • 16 Structured Probabilistic Models for Deep Learning
  • 17 Monte Carlo Methods
  • 18 Confronting the Partition Function
  • 19 Approximate Inference
  • 20 Deep Generative Models

Before the actual content we can find an introduction to the mathematics needed for the rest of the book, as well as some notation definitions. This introduction is as it should be: an executive summary, presenting subjects briefly, allowing reader to widen her knowledge with more specific texts if needed.

The following video is an interview of Dr Andrew Ng with one of the heroes of Deep Learning and authors of this book: Ian Goodfellow. Take a look, if you like the interview you will probably love the contents of the book.

Prerequisites

In order to be able to fully enjoy this deliciously written book, we advise that you have a strong algebra and calculus knowledge and that you are familiar at least with the main machine learning concepts: supervised and unsupervised training, metrics and loss functions, and the basic machine learning models like Linear and Logistic Regression.

About the book

Authors: Ian Goodfellow is a former Research Scientist at Google, now working as the Director of Machine Learning in the Special Projects Group at Apple. Yoshua Bengio is Professor of Computer Science at the University de Montreal, a top researcher in the world of Artificial Intelligence and one of the fathers of Deep Learning. Aaron Courville is Assistant Professor of Computer Science at the University de Montreal and expert in Machine Learning systems.

Pages: 800 pages

Publication: Last publication on 2017

Summary of Deep Learning by Goodfellow et al

Deep Learning by Ian Goodfellow et al is the go to book if you want to become and expert in the theory behind Deep Learning. It offers a very wide trip from the most basic stuff to the most complex topics, however this trip is smooth and well paved. Aside from the normal topics that interest students, (learning the theory of how things work), the book has some very clear and mind awakening discussions about topics that nowadays are la creme de la creme like autoencoders, generative models and Monte Carlo Methods.

If you want to become an expert in the theory of Machine Learning, and specifically Deep Learning, then this is the book for you. It is one of the best Machine Learning books for Deep Learning, and probably the best deep learning book if what you are looking for is understanding the maths about it all.

It is ultra comprehensive and will set you up for killing more practical books like Hands On Machine Learning with Tensorflow and Sckit Learn, or will make you understand implementations more thoroughly if you are already comfortable with the most practical aspects of Machine Learning.

You can buy it on Amazon:

If you are interested in Deep learning, and are looking for further resources, check our review of the Deep Learning Specialisation by Andrew Ng on Coursera!

Thank you very much for reading How to Learn Machine Learning, and have a great day!