Philip Schulz and I have designed a tutorial on variational inference and deep generative models for NLP audiences. All of our tutorial’s material is publicly available on github.

This is an extended edition I am presenting at Yandex NLP Week in Moscow.

Schedule

Check the branch yandex2019 for all modules

Day 1

Day 2

Day 3

Advices

For discrete latent variables:

Advanced topics

Beyond Gaussian posterior with normalising flows:

Beyond mean-field:

Beyond Gaussian prior:

More about posterior collapse:

Beyond KL divergence:

Beyond likelihood learning:

Beyond baselines:

Further reading

This is a list of papers you can use to kickstart your path to being an expert on DGMs.

Some people also asked me to list the techniques available to dealing with intermediate discrete representations (this is not an exhaustive list):

DGMs in NLP

Non-exhaustive list (let me know if you would like me to add a paper to this list):