2. Preliminaries

To get started with deep learning, we will need to develop a few basic skills. All machine learning is concerned with extracting information from data. So we will begin by learning the practical skills for storing, manipulating, and preprocessing data.

Moreover, machine learning typically requires working with large datasets, which we can think of as tables, where the rows correspond to examples and the columns correspond to attributes. Linear algebra gives us a powerful set of techniques for working with tabular data. We will not go too far into the weeds but rather focus on the basic of matrix operations and their implementation.

Additionally, deep learning is all about optimization. We have a model with some parameters and we want to find those that fit our data the best. Determining which way to move each parameter at each step of an algorithm requires a little bit of calculus, which will be briefly introduced. Fortunately, the autograd package automatically computes differentiation for us, and we will cover it next.

Next, machine learning is concerned with making predictions: what is the likely value of some unknown attribute, given the information that we observe? To reason rigorously under uncertainty we will need to invoke the language of probability.

In the end, the official documentation provides plenty of descriptions and examples that are beyond this book. To conclude the chapter, we will show you how to look up documentation for the needed information.

This book has kept the mathematical content to the minimum necessary to get a proper understanding of deep learning. However, it does not mean that this book is mathematics free. Thus, this chapter provides a rapid introduction to basic and frequently-used mathematics to allow anyone to understand at least most of the mathematical content of the book. If you wish to understand all of the mathematical content, further reviewing the online appendix on mathematics should be sufficient.

• 2.1. Data Manipulation
  • 2.1.1. Getting Started
  • 2.1.2. Operations
  • 2.1.3. Broadcasting Mechanism
  • 2.1.4. Indexing and Slicing
  • 2.1.5. Saving Memory
  • 2.1.6. Summary
  • 2.1.7. Exercises
• 2.2. Data Preprocessing
  • 2.2.1. Adding tablesaw dependencies to Jupyter notebook
  • 2.2.2. Reading the Dataset
  • 2.2.3. Handling Missing Data
  • 2.2.4. Conversion to the NDArray Format
  • 2.2.5. Summary
  • 2.2.6. Exercises
• 2.3. Linear Algebra
  • 2.3.1. Scalars
  • 2.3.2. Vectors
  • 2.3.3. Matrices
  • 2.3.4. NDArrays
  • 2.3.5. Basic Properties of NDArray Arithmetic
  • 2.3.6. Reduction
  • 2.3.7. Dot Products
  • 2.3.8. Matrix-Vector Products
  • 2.3.9. Matrix-Matrix Multiplication
  • 2.3.10. Norms
  • 2.3.11. More on Linear Algebra
  • 2.3.12. Summary
  • 2.3.13. Exercises
• 2.4. Calculus
  • 2.4.1. Derivatives and Differentiation
  • 2.4.2. Partial Derivatives
  • 2.4.3. Gradients
  • 2.4.4. Chain Rule
  • 2.4.5. Summary
  • 2.4.6. Exercises
• 2.5. Automatic Differentiation
  • 2.5.1. A Simple Example
  • 2.5.2. Backward for Non-Scalar Variables
  • 2.5.3. Detaching Computation
  • 2.5.4. Computing the Gradient of Java Control Flow
  • 2.5.5. Summary
  • 2.5.6. Exercises
• 2.6. Probability
  • 2.6.1. Basic Probability Theory
  • 2.6.2. Dealing with Multiple Random Variables
  • 2.6.3. Expectation and Variance
  • 2.6.4. Summary
  • 2.6.5. Exercises
• 2.7. Documentation
  • 2.7.1. Finding All the Functions and Classes in a Module
  • 2.7.2. Finding the Usage of Specific Functions and Classes
  • 2.7.3. Summary
  • 2.7.4. Exercises