Markov kernel

In probability theory, a Markov kernel (or stochastic kernel) is a map that plays the role, in the general theory of Markov processes, that the transition matrix does in the theory of Markov processes with a finite state space.^[1]

Formal definition

Let $(X,{\mathcal {A}})$, $(Y,{\mathcal {B}})$ be measurable spaces. A Markov kernel with source $(X,{\mathcal {A}})$ and target $(Y,{\mathcal {B}})$ is a map $\kappa \colon X\times {\mathcal {B}}\to [0,1]$ with the following properties:

1. The map $x\mapsto \kappa (x,B)$ is ${\mathcal {A}}$ - measureable for every $B\in {\mathcal {B}}$.
2. The map $B\mapsto \kappa (x,B)$ is a probability measure on $(Y,{\mathcal {B}})$ for every $x\in X$.

(i.e. It associates to each point $x\in X$ a probability measure $\kappa (x,.)$ on $(Y,{\mathcal {B}})$ such that, for every measurable set $B\in {\mathcal {B}}$, the map $x\mapsto \kappa (x,B)$ is measurable with respect to the $\sigma$-algebra ${\mathcal {A}}$.)^[2]

Examples

• Simple random walk: Take $X=Y=\mathbb {Z}$ and ${\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {Z} )$, then the Markov kernel $\kappa$ with

${\displaystyle \kappa (x,B)={\frac {1}{2}}\mathbf {1} _{B}(x-1)+{\frac {1}{2}}\mathbf {1} _{B}(x+1),\quad \forall x\in \mathbb {Z} ,\quad \forall B\in {\mathcal {P}}(\mathbb {Z} )}$,

describes the transition rule for the random walk on $\mathbb {Z}$, where $\mathbf {1}$ is the indicator function.

• Galton-Watson process: Take $X=Y=\mathbb {N}$, ${\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {N} )$, then

${\displaystyle \kappa (x,B)={\begin{cases}\mathbf {1} _{B}(0)&\quad x=0,\\P[\xi _{1}+\dots +\xi _{x}\in B]&\quad {\text{else,}}\\\end{cases}}}$

with i.i.d. random variables $\xi _{i}$.

• General Markov processes with finite state space: Take $X=Y$, ${\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(X)={\mathcal {P}}(Y)$ and $|X|=|Y|=n$, then the transition rule can be represented as a stochastic matrix $(K_{ij})_{1\leq i,j\leq n}$ with $\Sigma _{j\in Y}K_{ij}=1$ for every $i\in X$. In the convention of Markov kernels we write

$\kappa (i,B)=\Sigma _{j\in B}K_{ij},\quad \forall i\in X,\quad \forall B\in {\mathcal {B}}$.

• Construction of a Markov kernel: If $\nu$ is a finite measure on $(Y,{\mathcal {B}})$ and $k:X\times Y\to \mathbb {R} _{+}$ is a measurable function with respect to the product $\sigma$-algebra ${\mathcal {A}}\otimes {\mathcal {B}}$ and has the property

$\int _{X}k(x,y)\nu (\mathrm {d} y)=1,$

for all $x\in X$, then the mapping $\kappa :X\times {\mathcal {B}}\to [0,1]$

$\kappa (x,B)=\int _{B}k(x,y)\nu (\mathrm {d} y),$

defines a Markov kernel.^[3]

Properties

Semidirect product

Let $(X,{\mathcal {A}},P)$ be a probability space and $\kappa$ a Markov kernel from $(X,{\mathcal {A}})$ to some $(Y,{\mathcal {B}})$.

Then there exists a unique measure $Q$ on $(X\times Y,{\mathcal {A}}\otimes {\mathcal {B}})$, such that

$Q(A\times B)=\int _{A}\kappa (x,B)dP(x),\quad \forall A\in {\mathcal {A}},\quad \forall B\in {\mathcal {B}}$.

Regular conditional distribution

Let $(S,Y)$ be a Borel space, $X$ a $(S,Y)$ - valued random variable on the measure space $(\Omega ,{\mathcal {F}},P)$ and ${\mathcal {G}}\subseteq {\mathcal {F}}$ a sub-$\sigma$-algebra.

Then there exists a Markov kernel $\kappa$ from $(\Omega ,{\mathcal {G}})$ to $(S,Y)$, such that $\kappa (.,B)$ is a version of the conditional expectation $E[\mathbf {1} _{\{X\in B\}}|{\mathcal {G}}]$ for every $B\in Y$, i.e.

$P[X\in B|{\mathcal {G}}]=E[\mathbf {1} _{\{X\in B\}}|{\mathcal {G}}]=\kappa (\omega ,B),\quad P-a.s.\forall B\in {\mathcal {G}}$.

It is called regular conditional distribution of $X$ given ${\mathcal {G}}$ and is not uniquely defined.

References

1. Reiss, R. D. (1993). "A Course on Point Processes". Springer Series in Statistics. doi:10.1007/978-1-4613-9308-5. ISBN 978-1-4613-9310-8.
2. Klenke, Achim. Probability Theory: A Comprehensive Course (2 ed.). Springer. p. 180. doi:10.1007/978-1-4471-5361-0.
3. Erhan, Cinlar (2011). Probability and Stochastics. New York: Springer. pp. 37–38. ISBN 978-0-387-87858-4.

• Bauer, Heinz (1996), Probability Theory, de Gruyter, ISBN 3-11-013935-9

§36. Kernels and semigroups of kernels