Garfield.modules.compute_kl_reg_loss
- Garfield.modules.compute_kl_reg_loss(mu: Tensor, logstd: Tensor) Tensor[source]
Compute Kullback-Leibler divergence as per Kingma, D. P. & Welling, M. Auto-Encoding Variational Bayes. arXiv [stat.ML] (2013). Equation (10). This will encourage encodings to distribute evenly around the center of a continuous and complete latent space, producing similar (for points close in latent space) and meaningful content after decoding.
For detailed derivation, see https://stats.stackexchange.com/questions/318748/deriving-the-kl-divergence-loss-for-vaes.
- Parameters:
mu – Expected values of the normal latent distribution of each node (dim: n_nodes_current_batch, n_gps).
logstd – Log standard deviations of the normal latent distribution of each node (dim: n_nodes_current_batch, n_gps).
- Returns:
Kullback-Leibler divergence.
- Return type:
kl_reg_loss