Notation¶

The notation used throughout this book is summarized below.

Numbers¶

\(x\): A scalar
\(\mathbf{x}\): A vector
\(\mathbf{X}\): A matrix
\(\mathsf{X}\): A tensor
\(\mathbf{I}\): An identity matrix
\(x_i\), \([\mathbf{x}]_i\): The \(i^\mathrm{th}\) element of vector \(\mathbf{x}\)
\(x_{ij}\), \([\mathbf{X}]_{ij}\): The element of matrix \(\mathbf{X}\) at row \(i\) and column \(j\)

\(\mathcal{X}\): A set
\(\mathbb{Z}\): The set of integers
\(\mathbb{R}\): The set of real numbers
\(\mathbb{R}^n\): The set of \(n\)-dimensional vectors of real numbers
\(\mathbb{R}^{a\times b}\): The set of matrices of real numbers with \(a\) rows and \(b\) columns
\(\mathcal{A}\cup\mathcal{B}\): Union of sets \(\mathcal{A}\) and \(\mathcal{B}\)
\(\mathcal{A}\cap\mathcal{B}\): Intersection of sets \(\mathcal{A}\) and \(\mathcal{B}\)
\(\mathcal{A}\setminus\mathcal{B}\): Subtraction of set \(\mathcal{B}\) from set \(\mathcal{A}\)

\(f(\cdot)\): A function
\(\log(\cdot)\): The natural logarithm
\(\exp(\cdot)\): The exponential function
\(\mathbf{1}_\mathcal{X}\): The indicator function
\(\mathbf{(\cdot)}^\top\): Transpose of a vector or a matrix
\(\mathbf{X}^{-1}\): Inverse of matrix \(\mathbf{X}\)
\(\odot\): Hadamard (elementwise) product
\([\cdot, \cdot]\): Concatenation
\(\lvert \mathcal{X} \rvert\): Cardinality of set \(\mathcal{X}\)
\(\|\cdot\|_p\): \(\ell_p\) norm
\(\|\cdot\|\): \(\ell_2\) norm
\(\langle \mathbf{x}, \mathbf{y} \rangle\): Dot product of vectors \(\mathbf{x}\) and \(\mathbf{y}\)
\(\sum\): Series addition
\(\prod\): Series multiplication

\(\frac{dy}{dx}\): Derivative of \(y\) with respect to \(x\)
\(\frac{\partial y}{\partial x}\): Partial derivative of \(y\) with respect to \(x\)
\(\nabla_{\mathbf{x}} y\): Gradient of \(y\) with respect to \(\mathbf{x}\)
\(\int_a^b f(x) \;dx\): Definite integral of \(f\) from \(a\) to \(b\) with respect to \(x\)
\(\int f(x) \;dx\): Indefinite integral of \(f\) with respect to \(x\)

\(P(\cdot)\): Probability distribution
\(z \sim P\): Random variable \(z\) has probability distribution \(P\)
\(P(X \mid Y)\): Conditional probability of \(X \mid Y\)
\(p(x)\): Probability density function
\({E}_{x} [f(x)]\): Expectation of \(f\) with respect to \(x\)
\(X \perp Y\): Random variables \(X\) and \(Y\) are independent
\(X \perp Y \mid Z\): Random variables \(X\) and \(Y\) are conditionally independent given random variable \(Z\)
\(\mathrm{Var}(X)\): Variance of random variable \(X\)
\(\sigma_X\): Standard deviation of random variable \(X\)
\(\mathrm{Cov}(X, Y)\): Covariance of random variables \(X\) and \(Y\)
\(\rho(X, Y)\): Correlation of random variables \(X\) and \(Y\)
\(H(X)\): Entropy of random variable \(X\)
\(D_{\mathrm{KL}}(P\|Q)\): KL-divergence of distributions \(P\) and \(Q\)