Free Deep Learning Book (MIT Press)

The Deep Learning textbook is a resource intended to help students and practitioners enter the field of machine learning in general and deep learning in particular. The online version of the book is now complete and will remain available online for free.

Source for picture: click here

Content

• Table of Contents
• Acknowledgements
• Notation
• 1 Introduction
• Part I: Applied Math and Machine Learning Basics
  • 2 Linear Algebra
  • 3 Probability and Information Theory
  • 4 Numerical Computation
  • 5 Machine Learning Basics
• Part II: Modern Practical Deep Networks
  • 6 Deep Feedforward Networks
  • 7 Regularization for Deep Learning
  • 8 Optimization for Training Deep Models
  • 9 Convolutional Networks
  • 10 Sequence Modeling: Recurrent and Recursive Nets
  • 11 Practical Methodology
  • 12 Applications
• Part III: Deep Learning Research
  • 13 Linear Factor Models
  • 14 Autoencoders
  • 15 Representation Learning
  • 16 Structured Probabilistic Models for Deep Learning
  • 17 Monte Carlo Methods
  • 18 Confronting the Partition Function
  • 19 Approximate Inference
  • 20 Deep Generative Models
• Bibliography
• Index

For more information, click here. The book is also available on Amazon, and also here (MIT Press). 

Lectures

We plan to offer lecture slides accompanying all chapters of this book. We currently offer slides for only some chapters. If you are a course instructor and have your own lecture slides that are relevant, feel free to contact us if you would like to have your slides linked or mirrored from this site.

1. Introduction
   • Presentation of Chapter 1, based on figures from the book [.key] [.pdf]
   • Video of lecture by Ian and discussion of Chapter 1 at a reading group in San Francisco organized by Alena Kruchkova
2. Linear Algebra [.key][.pdf]
3. Probability and Information Theory [.key][.pdf]
4. Numerical Computation [.key] [.pdf] [youtube]
5. Machine Learning Basics [.key] [.pdf]
6. Deep Feedforward Networks [.key] [.pdf]
   • Video (.flv) of a presentation by Ian and a group discussion at a reading group at Google organized by Chintan Kaur.
7. Regularization for Deep Learning [.pdf] [.key]
8. Optimization for Training Deep Models
   • Gradient Descent and Structure of Neural Network Cost Functions [.key] [.pdf]
     These slides describe how gradient descent behaves on different kinds of cost function surfaces. Intuition for the structure of the cost function can be built by examining a second-order Taylor series approximation of the cost function. This quadratic function can give rise to issues such as poor conditioning and saddle points. Visualization of neural network cost functions shows how these and some other geometric features of neural network cost functions affect the performance of gradient descent.
   • Tutorial on Optimization for Deep Networks [.key] [.pdf]
     Ian's presentation at the 2016 Re-Work Deep Learning Summit. Covers Google Brain research on optimization, including visualization of neural network cost functions, Net2Net, and batch normalization.
   • Batch Normalization [.key] [.pdf]
   • Video of lecture / discussion: This video covers a presentation by Ian and group discussion on the end of Chapter 8 and entirety of Chapter 9 at a reading group in San Francisco organized by Taro-Shigenori Chiba.
9. Convolutional Networks
   • Convolutional Networks [.key][.pdf]
     A presentation summarizing Chapter 9, based directly on the textbook itself.
   • Video of lecture / discussion: This video covers a presentation by Ian and group discussion on the end of Chapter 8 and entirety of Chapter 9 at a reading group in San Francisco organized by Taro-Shigenori Chiba.
10. Sequence Modeling: Recurrent and Recursive Networks
    • Sequence Modeling [.pdf] [.key]
      A presentation summarizing Chapter 10, based directly on the textbook itself.
    • Video of lecture / discussion. This video covers a presentation by Ian and a group discussion of Chapter 10 at a reading group in San Francisco organized by Alena Kruchkova.
11. Practical Methodology [.key][.pdf] [youtube]
12. Applications [.key][.pdf]
13. Linear Factors [.key][.pdf]
14. Autoencoders [.key][.pdf]
15. Representation Learning [.key][.pdf]
16. Structured Probabilistic Models for Deep Learning[.key][.pdf]

Explainable Artificial Intelligence (XAI)

Dramatic success in machine learning has led to a torrent of Artificial Intelligence (AI) applications. Continued advances promise to produce autonomous systems that will perceive, learn, decide, and act on their own. However, the effectiveness of these systems is limited by the machine’s current inability to explain their decisions and actions to human users. The Department of Defense is facing challenges that demand more intelligent, autonomous, and symbiotic systems. Explainable AI—especially explainable machine learning—will be essential if future warfighters are to understand, appropriately trust, and effectively manage an emerging generation of artificially intelligent machine partners.

The Explainable AI (XAI) program aims to create a suite of machine learning techniques that:

• Produce more explainable models, while maintaining a high level of learning performance (prediction accuracy); and
• Enable human users to understand, appropriately trust, and effectively manage the emerging generation of artificially intelligent partners.

New machine-learning systems will have the ability to explain their rationale, characterize their strengths and weaknesses, and convey an understanding of how they will behave in the future. The strategy for achieving that goal is to develop new or modified machine-learning techniques that will produce more explainable models. These models will be combined with state-of-the-art human-computer interface techniques capable of translating models into understandable and useful explanation dialogues for the end user. Our strategy is to pursue a variety of techniques in order to generate a portfolio of methods that will provide future developers with a range of design options covering the performance-versus-explainability trade space.

 
Figure 1: XAI Concept

The XAI program will focus the development of multiple systems on addressing challenges problems in two areas: (1) machine learning problems to classify events of interest in heterogeneous, multimedia data; and (2) machine learning problems to construct decision policies for an autonomous system to perform a variety of simulated missions. These two challenge problem areas were chosen to represent the intersection of two important machine learning approaches (classification and reinforcement learning) and two important operational problem areas for the Department of Defense (intelligence analysis and autonomous systems).

XAI research prototypes will be tested and continually evaluated throughout the course of the program. At the end of the program, the final delivery will be a toolkit library consisting of machine learning and human-computer interface software modules that could be used to develop future explainable AI systems. After the program is complete, these toolkits would be available for further refinement and transition into defense or commercial applications.

Source: https://www.darpa.mil/program/explainable-artificial-intelligence