Introduction to Algorithmic Information Theory, Belief Propagation, Graph Signal Processing, and Deep Reinforcement Learning

Information Information Theory

Compression (Entropy):

$H(X) = -\sum_{x} p(x)\log_{2}p(x)$

Capacity:

$\begin{align*} SIR &= I(X{;}Y) &= H(X) - H(X|Y) \end{align*}$

Belief Propagation: We can use Belief propagation to calculate the channel Capacity of some Markov chain...and so on