Position in the series

latent variable -> ELBO

Outline

In this section, we will answer these questions:

  1. Why introduce latent variables?
  2. Marginal likelihood and the intractable integral.
  3. Variational posterior $q(z \mid x)$.
  4. Jensen’s inequality and ELBO.
  5. Reconstruction term and KL term.
  6. ELBO as a probabilistic modeling principle.

Notes

Since the distribution of real-world data is highly complex, we introduce latent variables $z$ to explain the observed data $x$. Thus, we do not model the data distribution directly as:

\[p_\theta(x)\]

Instead, we assume the data is generated through latent variables $z$:

\[\begin{aligned} z &\sim p(z), \\ x &\sim p_\theta(x \mid z). \end{aligned}\]

Finally,

\[p_\theta(x) = \int p_\theta(x \mid z)p(z)\,dz\]

In this place, $z$ is the latent variable.