How is the negative log-likelihood of the data under the RealNVP model calculated?
The negative log-likelihood of the data under the RealNVP model is calculated using the formula: -log pX(x) = -log pZ(z) - log det(∂z/∂x). This involves the log probability of the transformed data under a Gaussian distribution and the log determinant of the Jacobian of the transformation.
In the RealNVP model, the negative log-likelihood of the data is calculated by transforming the data into a latent space using a series of invertible transformations. The formula used is -log pX(x) = -log pZ(z) - log det(∂z/∂x), where pZ(z) is the probability of the transformed data under a standard Gaussian distribution. The log determinant of the Jacobian, log det(∂z/∂x), accounts for the change in volume induced by the transformation. This approach allows for an exact and tractable computation of the data distribution, leveraging the properties of the RealNVP transformations to maintain a simple Jacobian determinant and invertibility.
Key points
- Negative log-likelihood is calculated as -log pX(x) = -log pZ(z) - log det(∂z/∂x).
- pZ(z) is a standard Gaussian distribution.
- The log determinant of the Jacobian accounts for volume changes.
- RealNVP transformations ensure invertibility and simple Jacobian calculation.
- This method allows for exact, tractable computation of the data distribution.
Related questions
Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play
David Foster;
Second Edition · O’Reilly Media, Inc.