Stochastic Processes

1. Random Processes: Concepts and Basic Types

• 1.1. Second-Order Processes

  A random process $\{X(t), t \in T\}$ is a second-order process if its mean $E[X(t)]$ and variance $D[X(t)]$ exist for all $t \in T$. By the Cauchy-Schwarz inequality, the autocorrelation function $R_X(t, s) = E[X(t)X(s)]$ and autocovariance function $C_X(t, s) = E[(X(t) - E[X(t)])(X(s) - E[X(s)])]$ are guaranteed to exist.

• 1.2. Joint Autocorrelation and Cross-Correlation

  The cross-correlation between two random processes $X(t)$ and $Y(t)$ is defined as $R_{XY}(t, s) = E[X(t)Y(s)]$. If they are jointly wide-sense stationary, it depends only on the time difference $\tau = t - s$:

  $$ R_{XY}(t, s) = R_{XY}(t - s) $$

  Cross-correlation satisfies the symmetry property: $R_{XY}(\tau) = R_{YX}(-\tau)$. If the processes are real-valued, $R_{XY}(\tau) = R_{YX}(-\tau)$.

• 1.3. Stationary Processes (Strict-sense and Wide-sense)
  • Strict-Sense Stationary (SSS): The joint distribution is invariant under any shift in time: $$ F_X(x_1, \dots, x_n; t_1, \dots, t_n) = F_X(x_1, \dots, x_n; t_1 + \tau, \dots, t_n + \tau) $$
  • Wide-Sense Stationary (WSS): The mean is constant and the autocorrelation depends only on time difference: $$ E[X(t)] = \mu_X = \text{constant}, \quad R_X(t, s) = R_X(t - s) $$
• 1.4. Properties of the Autocorrelation Function
  1. Symmetry: $R_X(\tau) = R_X(-\tau)$.
  2. Boundedness: $|R_X(\tau)| \le R_X(0) = E[X^2(t)]$.
  3. Relation to Autocovariance: $$ C_X(t, s) = R_X(t, s) - E[X(t)]E[X(s)] $$

2. Mean Ergodicity

A WSS process $\{X(t)\}$ is mean-ergodic if its time average converges to its ensemble average in the mean-square sense as $T \to \infty$:

$$ \hat{\mu}_T = \frac{1}{T} \int_{-T/2}^{T/2} X(t) dt \xrightarrow{\text{m.s.}} E[X(t)] = \mu_X $$

A necessary and sufficient condition for mean ergodicity is:

$$ \lim_{T \to \infty} \frac{1}{T} \int_{-T}^T \left(1 - \frac{|\tau|}{T}\right) C_X(\tau) d\tau = 0 $$

A sufficient condition is that $\lim_{\tau \to \infty} C_X(\tau) = 0$.

3. Karhunen-Loève (KL) Expansion

The KL expansion represents a continuous-time second-order random process $X(t)$ over an interval $[a, b]$ as an infinite linear combination of orthogonal coordinate functions:

$$ X(t) = \sum_{n=1}^{\infty} X_n \phi_n(t) $$

where $\{\phi_n(t)\}$ are deterministic orthonormal eigenfunctions satisfying the integral equation:

$$ \int_a^b R_X(t, s) \phi_n(s) ds = \lambda_n \phi_n(t) $$

The coefficients $X_n$ are uncorrelated random variables given by:

$$ X_n = \int_a^b X(t) \phi_n^*(t) dt $$

and satisfy $E[X_n X_m^*] = \lambda_n \delta_{nm}$. The autocorrelation function can be expanded as:

$$ R_X(t, s) = \sum_{n=1}^{\infty} \lambda_n \phi_n(t) \phi_n^*(s) $$

4. Linear Time-Invariant (LTI) Systems with Random Inputs

When a WSS process $X(t)$ is passed through a stable LTI system with impulse response $h(t)$ and transfer function $H(\omega)$, the output $Y(t) = h(t) * X(t)$ is also WSS.

• Output Mean: $$ E[Y(t)] = E[X(t)] \int_{-\infty}^{\infty} h(t) dt = E[X(t)] H(0) $$
• Power Spectral Density (PSD) Relations: $$ S_Y(\omega) = |H(\omega)|^2 S_X(\omega) $$ $$ S_{YX}(\omega) = H(\omega) S_X(\omega), \quad S_{XY}(\omega) = \overline{H(\omega)} S_X(\omega) $$

5. Hilbert Transform and Analytic Signals

The Hilbert transform of a random process $X(t)$ is defined as:

$$ \hat{X}(t) = X(t) * \frac{1}{\pi t} = \frac{1}{\pi} \int_{-\infty}^{\infty} \frac{X(\tau)}{t - \tau} d\tau $$

The equivalent filter has transfer function $H(\omega) = -j \text{sgn}(\omega)$.

• Analytic Signal: $Y(t) = X(t) + j\hat{X}(t)$.
• Autocorrelation of Analytic Signal: $$ R_Y(\tau) = 2 R_X(\tau) + 2j \hat{R}_{X}(\tau) $$
• PSD of Analytic Signal: $$ S_Y(\omega) = 2 S_X(\omega) [1 + \text{sgn}(\omega)] = 4 S_X(\omega) u(\omega) $$

6. In-Phase and Quadrature (I/Q) Components of Bandpass Processes

A bandpass process $Y(t)$ centered at carrier frequency $\omega_c$ can be decomposed into in-phase $X_r(t)$ and quadrature $X_i(t)$ components:

$$ Y(t) = X_r(t) \cos(\omega_c t) - X_i(t) \sin(\omega_c t) $$ $$ \hat{Y}(t) = X_r(t) \sin(\omega_c t) + X_i(t) \cos(\omega_c t) $$

where $X_r(t)$ and $X_i(t)$ are lowpass processes defined as:

$$ X_r(t) = Y(t)\cos(\omega_c t) + \hat{Y}(t)\sin(\omega_c t) $$ $$ X_i(t) = \hat{Y}(t)\cos(\omega_c t) - Y(t)\sin(\omega_c t) $$

If $Y(t)$ is zero-mean and WSS, then $X_r(t)$ and $X_i(t)$ are jointly WSS with the following properties:

1. $E[X_r(t)] = E[X_i(t)] = 0$.
2. $R_{X_r}(\tau) = R_{X_i}(\tau) = R_Y(\tau)\cos(\omega_c \tau) + \hat{R}_Y(\tau)\sin(\omega_c \tau)$.
3. $R_{X_r X_i}(\tau) = R_Y(\tau)\sin(\omega_c \tau) - \hat{R}_Y(\tau)\cos(\omega_c \tau) = -R_{X_i X_r}(\tau)$.
4. $S_{X_r}(\omega) = S_{X_i}(\omega) = [S_Y(\omega - \omega_c) + S_Y(\omega + \omega_c)]_{\text{LP}}$.
5. $S_{X_r X_i}(\omega) = j[S_Y(\omega - \omega_c) - S_Y(\omega + \omega_c)]_{\text{LP}}$.

7. Gaussian Processes

1. Multivariate Density Function:

   A vector of random variables $\bm{X} = (X_1, \dots, X_n)^T$ is Gaussian if its joint probability density is:

   $$ f_{\bm{X}}(\bm{x}) = \frac{1}{(2\pi)^{n/2} |\bm{\Sigma}|^{1/2}} \exp\left( -\frac{1}{2}(\bm{x} - \bm{\mu})^T \bm{\Sigma}^{-1} (\bm{x} - \bm{\mu}) \right) $$

   where $\bm{\mu}$ is the mean vector and $\bm{\Sigma}$ is the positive-definite covariance matrix.

2. Characteristic Function: $$ \Phi_{\bm{X}}(\bm{\omega}) = E\{ e^{j \bm{\omega}^T \bm{X}} \} = \exp\left( j \bm{\omega}^T \bm{\mu} - \frac{1}{2} \bm{\omega}^T \bm{\Sigma} \bm{\omega} \right) $$
3. Moments property: A Gaussian process is completely specified by its first and second moments (mean and autocorrelation).
4. Linear Transformation: If $\bm{X}$ is Gaussian and $\bm{Y} = \bm{C}\bm{X} + \bm{d}$, then $\bm{Y}$ is also Gaussian.
5. Conditional Gaussian Distribution:

   If $\bm{X} = (\bm{X}_1, \bm{X}_2)^T$ is Gaussian, the conditional distribution of $\bm{X}_1$ given $\bm{X}_2$ is also Gaussian:

   $$ \bm{X}_1 \mid \bm{X}_2 \sim \mathcal{N}\left( \bm{\mu}_1 + \bm{\Sigma}_{12}\bm{\Sigma}_{22}^{-1}(\bm{x}_2 - \bm{\mu}_2), \, \bm{\Sigma}_{11} - \bm{\Sigma}_{12}\bm{\Sigma}_{22}^{-1}\bm{\Sigma}_{21} \right) $$
6. Uncorrelatedness implies Independence: For Gaussian variables, they are independent if and only if they are uncorrelated ($\bm{\Sigma}_{12} = \bm{0}$).

8. Gauss-Markov Processes

A Gaussian process $X(t)$ is Markov if and only if for any $t_1 < t_2 < t_3$, the correlation satisfies the relation:

$$ C_X(t_1, t_3) = \frac{C_X(t_1, t_2) C_X(t_2, t_3)}{C_X(t_2, t_2)} $$

which implies the covariance factorizes as $C_X(t, s) = h(\min(t,s)) g(\max(t,s))$.

9. Poisson Processes

A Poisson process $\{N(t), t \ge 0\}$ with intensity $\lambda > 0$ is a discrete-state, continuous-time process representing count statistics of independent events:

$$ P(N(t) = n) = \frac{(\lambda t)^n}{n!} e^{-\lambda t}, \quad n = 0, 1, \dots $$

• Mean and Variance: $E[N(t)] = D[N(t)] = \lambda t$.
• Autocorrelation Function: $$ R_N(t, s) = E[N(t)N(s)] = \lambda^2 t s + \lambda \min(t, s) $$
• Autocovariance Function: $$ C_N(t, s) = \lambda \min(t, s) $$

10. Poisson Process Arrival Times and Waiting Times

1. Inter-Arrival Times ($T_n$):

   The time intervals between consecutive events $T_n = S_n - S_{n-1}$ are independent, identically distributed (i.i.d.) exponential random variables:

   $$ f_{T_n}(t) = \lambda e^{-\lambda t}, \quad t \ge 0, \quad E[T_n] = \frac{1}{\lambda}, \quad D[T_n] = \frac{1}{\lambda^2} $$
2. Waiting Times ($S_n$):

   The arrival time of the $n$-th event $S_n = \sum_{i=1}^n T_i$ follows a Gamma (Erlang) distribution:

   $$ f_{S_n}(t) = \frac{\lambda^n t^{n-1}}{(n-1)!} e^{-\lambda t}, \quad t \ge 0 $$
3. Conditional Distribution of Arrival Times:

   Given that $N(t) = n$ events occurred in the interval $[0, t]$, the arrival times $\{S_1, \dots, S_n\}$ are distributed as the order statistics of $n$ independent random variables uniformly distributed on $[0, t]$:

   $$ f_{S_1, \dots, S_n \mid N(t)=n}(u_1, \dots, u_n) = \frac{n!}{t^n}, \quad 0 \le u_1 \le \dots \le u_n \le t $$

11. Compound Poisson Processes

A compound Poisson process $\{Y(t), t \ge 0\}$ is defined as $Y(t) = \sum_{k=1}^{N(t)} H_k$, where $\{N(t)\}$ is a Poisson process and $\{H_k\}$ are i.i.d. random variables independent of $N(t)$.

• Mean: $E[Y(t)] = \lambda t E[H_k]$.
• Variance: $$ D[Y(t)] = \lambda t E[H_k^2] $$
• Characteristic Function: $$ G_Y(u) = e^{\lambda t [G_H(u) - 1]} $$

12. Non-Homogeneous Poisson Processes

If the rate of arrival varies with time as $\lambda(t)$, the process has independent increments, and the count follows:

$$ P(N(t) - N(s) = n) = \frac{\Lambda(s, t)^n}{n!} e^{-\Lambda(s, t)}, \quad \text{where } \Lambda(s, t) = \int_s^t \lambda(u) du $$

13. Discrete-Time Markov Chains (DTMC)

1. Markov Property: $$ P(X_{n+1} = j \mid X_n = i, X_{n-1} = i_{n-1}, \dots, X_0 = i_0) = P(X_{n+1} = j \mid X_n = i) = P_{ij} $$
2. Chapman-Kolmogorov Equations: $$ P_{ij}^{(m+n)} = \sum_{k} P_{ik}^{(m)} P_{kj}^{(n)} $$
3. State Classification:
   • Recurrent: A state is recurrent if the probability of eventually returning to it is 1 ($f_{ii} = 1$). - If mean return time $\mu_i = E[T_{ii}] < \infty$, the state is positive recurrent. - If $\mu_i = \infty$, it is null recurrent.
   • Transient: A state is transient if $f_{ii} < 1$, meaning $\sum_{n=1}^{\infty} P_{ii}^{(n)} < \infty$.
4. Stationary Distribution ($\bm{\pi}$):

   For an irreducible, aperiodic Markov chain, the stationary distribution satisfies the balance equations:

   $$ \pi_j = \sum_{k} \pi_k P_{kj}, \quad \sum_{j} \pi_j = 1 $$

   The limiting distribution is given by $P_{ij}^{(n)} \to \pi_j = 1/\mu_j$ as $n \to \infty$.

14. Common Autocorrelation and Power Spectral Density Pairs