.. _chap_notation:

Notation
========


The notation used throughout this book is summarized below.

Numbers
-------

-  :math:`x`: A scalar
-  :math:`\mathbf{x}`: A vector
-  :math:`\mathbf{X}`: A matrix
-  :math:`\mathsf{X}`: A tensor
-  :math:`\mathbf{I}`: An identity matrix
-  :math:`x_i`, :math:`[\mathbf{x}]_i`: The :math:`i^\mathrm{th}`
   element of vector :math:`\mathbf{x}`
-  :math:`x_{ij}`, :math:`[\mathbf{X}]_{ij}`: The element of matrix
   :math:`\mathbf{X}` at row :math:`i` and column :math:`j`

Set Theory
----------

-  :math:`\mathcal{X}`: A set
-  :math:`\mathbb{Z}`: The set of integers
-  :math:`\mathbb{R}`: The set of real numbers
-  :math:`\mathbb{R}^n`: The set of :math:`n`-dimensional vectors of
   real numbers
-  :math:`\mathbb{R}^{a\times b}`: The set of matrices of real numbers
   with :math:`a` rows and :math:`b` columns
-  :math:`\mathcal{A}\cup\mathcal{B}`: Union of sets :math:`\mathcal{A}`
   and :math:`\mathcal{B}`
-  :math:`\mathcal{A}\cap\mathcal{B}`: Intersection of sets
   :math:`\mathcal{A}` and :math:`\mathcal{B}`
-  :math:`\mathcal{A}\setminus\mathcal{B}`: Subtraction of set
   :math:`\mathcal{B}` from set :math:`\mathcal{A}`

Functions and Operators
-----------------------

-  :math:`f(\cdot)`: A function
-  :math:`\log(\cdot)`: The natural logarithm
-  :math:`\exp(\cdot)`: The exponential function
-  :math:`\mathbf{1}_\mathcal{X}`: The indicator function
-  :math:`\mathbf{(\cdot)}^\top`: Transpose of a vector or a matrix
-  :math:`\mathbf{X}^{-1}`: Inverse of matrix :math:`\mathbf{X}`
-  :math:`\odot`: Hadamard (elementwise) product
-  :math:`[\cdot, \cdot]`: Concatenation
-  :math:`\lvert \mathcal{X} \rvert`: Cardinality of set
   :math:`\mathcal{X}`
-  :math:`\|\cdot\|_p`: :math:`\ell_p` norm
-  :math:`\|\cdot\|`: :math:`\ell_2` norm
-  :math:`\langle \mathbf{x}, \mathbf{y} \rangle`: Dot product of
   vectors :math:`\mathbf{x}` and :math:`\mathbf{y}`
-  :math:`\sum`: Series addition
-  :math:`\prod`: Series multiplication

Calculus
--------

-  :math:`\frac{dy}{dx}`: Derivative of :math:`y` with respect to
   :math:`x`
-  :math:`\frac{\partial y}{\partial x}`: Partial derivative of
   :math:`y` with respect to :math:`x`
-  :math:`\nabla_{\mathbf{x}} y`: Gradient of :math:`y` with respect to
   :math:`\mathbf{x}`
-  :math:`\int_a^b f(x) \;dx`: Definite integral of :math:`f` from
   :math:`a` to :math:`b` with respect to :math:`x`
-  :math:`\int f(x) \;dx`: Indefinite integral of :math:`f` with respect
   to :math:`x`

Probability and Information Theory
----------------------------------

-  :math:`P(\cdot)`: Probability distribution
-  :math:`z \sim P`: Random variable :math:`z` has probability
   distribution :math:`P`
-  :math:`P(X \mid Y)`: Conditional probability of :math:`X \mid Y`
-  :math:`p(x)`: Probability density function
-  :math:`{E}_{x} [f(x)]`: Expectation of :math:`f` with respect to
   :math:`x`
-  :math:`X \perp Y`: Random variables :math:`X` and :math:`Y` are
   independent
-  :math:`X \perp Y \mid Z`: Random variables :math:`X` and :math:`Y`
   are conditionally independent given random variable :math:`Z`
-  :math:`\mathrm{Var}(X)`: Variance of random variable :math:`X`
-  :math:`\sigma_X`: Standard deviation of random variable :math:`X`
-  :math:`\mathrm{Cov}(X, Y)`: Covariance of random variables :math:`X`
   and :math:`Y`
-  :math:`\rho(X, Y)`: Correlation of random variables :math:`X` and
   :math:`Y`
-  :math:`H(X)`: Entropy of random variable :math:`X`
-  :math:`D_{\mathrm{KL}}(P\|Q)`: KL-divergence of distributions
   :math:`P` and :math:`Q`

Complexity
----------

-  :math:`\mathcal{O}`: Big O notation