KL_divergence = nan | Learn AI Together | Page 1

bold light Jan 11, 2023, 11:43 PM

#

Hi, i have two distributions type Bernoulli, but the KL-divergence is equal to nan. Do you have any idea ?

grand finch Jan 14, 2023, 4:45 PM

#

Referring here https://stackoverflow.com/questions/59736801/why-do-i-get-a-nan-when-computing-the-kl-divergence-between-two-onehotcategorica - it might be because of log(0) or the 0/0 operation? Have you tried adding a small epsilon to see if you can get a different output?

Stack Overflow

I try to calculate the KL-Divergence between two OneHotCategorical distributions with the following code:

posterior = tfd.OneHotCategorical(probs=[0., 0., 0., 0., 0., 0., 0., 1.])
prior = tfd.

bold light Jan 14, 2023, 5:06 PM

#

the problem is that i use distrib.log_prob(y) with distrib a bernoulli

#

I can add a small epsilon to y but not to the distribtuion

#

thats where i'm stuck

#KL_divergence = nan