Appendix: Some Maths - Data Science for Biotechnology students

Linear algebra¶

Row Vector¶

A row vector is a 1-dimensional array consisting of a single row of elements. For example, a row vector with three elements can be written as:

\mathbf{r} = [r_1, r_2, r_3]

(1)

Column Vector¶

A column vector is a 1-dimensional array consisting of a single column of elements. It can be thought of as a matrix with one column. For instance, a column vector with three elements appears as:

\mathbf{c} = \begin{bmatrix} c_1 \\ c_2 \\ c_3 \end{bmatrix}

(2)

Matrix¶

A matrix is a rectangular array of numbers arranged in rows and columns. For example, a matrix with two rows and three columns is shown as:

\mathbf{A} = \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \end{bmatrix}

(3)

Transpose Operator¶

The transpose of a matrix is obtained by swapping its rows with its columns. The transpose of matrix $\mathbf{A}$ is denoted $\mathbf{A}^T$ . For the given matrix $\mathbf{A}$ , the transpose is:

\mathbf{A}^T = \begin{bmatrix} a_{11} & a_{21} \\ a_{12} & a_{22} \\ a_{13} & a_{23} \end{bmatrix}

(4)

Multiplication between Vectors¶

Vector multiplication can result in either a scalar or a matrix:

Dot product: Multiplication of the transpose of a column vector $\mathbf{a}$ with another column vector $\mathbf{b}$ results in a scalar. This is also known as the inner product:

\mathbf{a}^T \mathbf{b} = a_1b_1 + a_2b_2 + a_3b_3

(5)

Outer product: The multiplication of a column vector $\mathbf{a}$ by the transpose of another column vector $\mathbf{b}$ results in a matrix:

\mathbf{a} \mathbf{b}^T = \begin{bmatrix} a_1b_1 & a_1b_2 & a_1b_3 \\ a_2b_1 & a_2b_2 & a_2b_3 \\ a_3b_1 & a_3b_2 & a_3b_3 \end{bmatrix}

(6)

Matrix Multiplication¶

The product of an $n \times k$ matrix $\mathbf{A}$ and a $k \times p$ matrix $\mathbf{B}$ is an $n \times p$ matrix $\mathbf{C}$ ; the number of columns of $\mathbf{A}$ has to equal the number of rows of $\mathbf{B}$ . Each element $c_{ij}$ of $\mathbf{C}$ is computed as the dot product of the $i$ -th row of $\mathbf{A}$ and the $j$ -th column of $\mathbf{B}$ :

c_{ij} = \sum_{k} a_{ik} b_{kj}

(7)

L2-norm (Euclidean norm)¶

The L2-norm of a vector $\mathbf{x} = [x_1,\dots,x_n]^T$ (also called the Euclidean norm) measures its length and is defined as:

\|\mathbf{x}\|_2 = \sqrt{\sum_{i=1}^n x_i^2}.

(8)

Projection¶

The projection of vector $\mathbf{u}$ onto vector $\mathbf{v}$ is given by:

\text{proj}_{\mathbf{v}} \mathbf{u} = \frac{\mathbf{u}^T\mathbf{v}}{\mathbf{v}^T\mathbf{v}} \mathbf{v}

(9)

This represents the orthogonal projection of $\mathbf{u}$ in the direction of $\mathbf{v}$ .

import numpy as np
import matplotlib.pyplot as plt

u = np.array([1, 3])
v = np.array([3, 1])

# Projection of u onto v, and the component of u orthogonal to v
proj_u_on_v = np.dot(u, v) / np.dot(v, v) * v
orthogonal_component = u - proj_u_on_v

# Plot u, v and the projection of u onto v
plt.figure(figsize=(6, 6))
plt.quiver(0, 0, u[0], u[1], angles='xy', scale_units='xy', scale=1, color='r', label=r'$\mathbf{u}=(1,3)^T$')
plt.quiver(0, 0, v[0], v[1], angles='xy', scale_units='xy', scale=1, color='b', label=r'$\mathbf{v}=(3,1)^T$')
plt.quiver(0, 0, proj_u_on_v[0], proj_u_on_v[1], angles='xy', scale_units='xy', scale=1, color='g', label=r'proj$_{\mathbf{v}} \mathbf{u}$')

# Plot orthogonal line as a dotted line segment
end_point = proj_u_on_v + orthogonal_component
plt.plot([proj_u_on_v[0], end_point[0]], [proj_u_on_v[1], end_point[1]], 'purple', linestyle='dotted', label='Orthogonal Component')

# Set plot limits and aspect
plt.xlim(0, 4)
plt.ylim(0, 4)
plt.gca().set_aspect('equal', adjustable='box')

# Add a grid, legend, and labels
plt.grid(True)
plt.legend()
plt.title(r'Projection of Vector $\mathbf{u}$ onto Vector $\mathbf{v}$')
plt.xlabel('X axis')
plt.ylabel('Y axis')
plt.show()

Eigenvalue and Eigenvector¶

An eigenvalue $\lambda$ and its corresponding eigenvector $\mathbf{v} \neq \mathbf{0}$ of a square matrix $\mathbf{A}$ satisfy the equation:

\mathbf{A} \mathbf{v} = \lambda \mathbf{v}

(10)

Gradient¶

The gradient of a multivariable function $f(\mathbf{x})$ is a vector of partial derivatives, which points in the direction of the steepest ascent of $f$ :

\nabla f(\mathbf{x}) = \begin{bmatrix} \frac{\partial f}{\partial x_1} \\ \frac{\partial f}{\partial x_2} \\ \vdots \\ \frac{\partial f}{\partial x_n} \end{bmatrix}

(11)

Basics of Probability¶

Descriptive Statistics¶

Mean¶

The mean of a set of numbers is the average of all the values. For a set $X = \{x_1, x_2, \dots, x_n\}$ , the sample mean $\bar{x}$ is calculated as:

\bar{x} = \frac{1}{n} \sum_{i=1}^n x_i

(12)

Variance¶

The variance measures the spread of a set of numbers around their mean. For a sample $X$ with sample mean $\bar{x}$ , the maximum likelihood estimate of the variance $\sigma^2$ (assuming normally distributed data) is

\hat{\sigma}^2_{\text{MLE}} = \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2 .

(13)

Because the same data are used to estimate the mean, this estimate is biased (it tends to underestimate the variance). Dividing by $n-1$ instead (Bessel’s correction) gives the unbiased estimate

s^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})^2 .

(14)

Covariance¶

Covariance is a measure of how much two random variables change together. For paired samples $x_1, \dots, x_n$ and $y_1, \dots, y_n$ of the variables $X$ and $Y$ , with sample means $\bar{x}$ and $\bar{y}$ , the maximum likelihood estimate of the covariance is

\widehat{\text{cov}}(X, Y) = \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y}) ,

(15)

while the unbiased estimate uses $n-1$ :

s_{XY} = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y}) .

(16)

Probability Distributions¶

Probability Density Function (PDF)¶

The probability density function (pdf) $p(x)$ of a continuous random variable is a function that describes how densely the probability is concentrated around each value of the random variable; the probability of any single exact value is zero. The area under the pdf curve between two values represents the probability of the variable falling within that range. A discrete random variable is instead described by its probabilities $\Pr(X = x)$ of taking each value $x$ .

Cumulative Distribution Function (CDF)¶

The cumulative distribution function (cdf) of a random variable $X$ gives the probability that $X$ will take a value less than or equal to $x$ . It is defined as:

F(x) = \Pr(X \leq x)

(17)

Uniform Distribution¶

The uniform distribution is a probability distribution where every outcome is equally likely over a given interval $[a, b]$ . Its pdf is:

p(x) = \frac{1}{b-a} \quad \text{for } x \in [a, b], \quad \text{and } p(x) = 0 \text{ otherwise}

(18)

Normal Distribution¶

The normal distribution is a probability distribution that is symmetric about its mean $\mu$ , with values near the mean being more frequent than values far from it. With standard deviation $\sigma$ , its pdf is given by:

p(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x - \mu)^2}{2\sigma^2}}

(19)

Expectation Value¶

The expectation value of a random variable is the long-run average value of repetitions of the experiment it represents. For a discrete random variable $X$ it is given by the sum over its possible values, and for a continuous random variable with pdf $p(x)$ by the corresponding integral:

E[X] = \sum_{x} x \cdot \Pr(X = x) \quad \text{(discrete)} \quad \text{or} \quad E[X] = \int_{-\infty}^{\infty} x \cdot p(x)\, dx \quad \text{(continuous)}

(20)

Relationships Between Random Variables¶

Joint Probability Distributions¶

A joint probability distribution describes the probability (or density) of two or more random variables taking specific values together. For two variables $X$ and $Y$ ,

\Pr(X = x, Y = y) \quad \text{or} \quad p(x, y)

(21)

denotes their joint distribution. From it, we can obtain the marginal distributions (see below) and conditional relationships such as

\Pr(Y = y \mid X = x) = \frac{\Pr(X = x, Y = y)}{\Pr(X = x)}, \quad p(y \mid x) = \frac{p(x, y)}{p(x)}.

(22)

Marginal Probability Distributions¶

A marginal probability distribution gives the probability or density of a subset of variables, obtained by summing or integrating out the others from a joint distribution. For example, from a joint distribution of $X$ and $Y$ :

\Pr(X = x) = \sum_y \Pr(X = x, Y = y), \quad p(x) = \int p(x, y)\,dy.

(23)

This process is called marginalization. The resulting marginal describes the distribution of $X$ alone, regardless of $Y$ . The marginal distribution of the data reappears below, as the denominator of Bayes’ theorem.

Probabilistic Inference¶

Likelihood¶

The likelihood function expresses how well different parameter values explain the observed data. For continuous data, it is written as $L(\theta \mid x) = p(x \mid \theta),$ and for discrete data as $L(\theta \mid x) = \Pr(x \mid \theta).$

Although the likelihood has the same mathematical form as a probability or density function, its interpretation differs: in the likelihood, the data $x$ are fixed and $\theta$ varies. The likelihood is not a probability distribution over $\theta$ ; its role is to compare parameter values in light of the data.

Maximum Likelihood¶

The maximum likelihood estimate (MLE) is the parameter value that maximizes the likelihood function:

\hat{\theta}_{\text{MLE}} = \arg\max_{\theta} L(\theta \mid x).

(24)

It identifies the parameter that makes the observed data most probable (or most dense) under the model. MLEs are widely used because, under general conditions, they are consistent, asymptotically efficient, and asymptotically normal. In practice, the likelihood is often maximized numerically using gradient-based or expectation–maximization algorithms.

Prior Probability¶

The prior probability $\Pr(\theta)$ (or density $p(\theta)$ ) represents our beliefs about parameter values before any data are observed. Priors can encode earlier experimental knowledge, theoretical limits, or subjective expectations. When little information is available, weakly informative or noninformative priors are used to prevent the prior from dominating the inference. In continuous models, the prior is a density $p(\theta)$ satisfying $\int p(\theta)\,d\theta = 1$ ; in discrete models, it is a probability mass function $\Pr(\theta)$ summing to one.

Posterior Probability¶

After observing data $x$ , the prior is updated using Bayes’ theorem:

p(\theta \mid x) = \frac{p(x \mid \theta)\,p(\theta)}{p(x)} \quad \text{or} \quad \Pr(\theta \mid x) = \frac{\Pr(x \mid \theta)\,\Pr(\theta)}{\Pr(x)}.

(25)

The resulting posterior distribution combines prior information with the information from the data through the likelihood. The denominator,

p(x) = \int p(x \mid \theta)\,p(\theta)\,d\theta,

(26)

(or the corresponding sum for discrete variables) is the marginal distribution of the data, also called the marginal likelihood or evidence, and ensures that the posterior integrates or sums to one. The posterior forms the basis for Bayesian inference, from which we can compute summaries such as the posterior mean, variance, or mode.