Welcome to 520A!

Instructor: Alexandre Bouchard-Côté (Alex)

Note: recorded lectures and slides will be posted on the course website

During the lecture:

• Ask questions!!! Two methods:
  • Write your question in the chat window (can be anonymous if you prefer)
  • Write “question!” in the chat window while you unmute and I’ll finish my sentence and let you speak
  • Valid questions include “too fast!”, “can you repeat that”, etc
• If you are able to and comfortable with it, please turn on your video! It help to be able to see facial expression to assess too fast/too slow

Outline

Course outline:

• a Bayesian “bootcamp” facilitated by probabilistic programming languages
• more specialized topics, with an emphasis on MCMC
• a final project in teams or individually

Today:

• Course logistics
• What is Bayesian analysis?
• Introductory example

Prerequisite

• Background in probability (e.g. Stat 302 or equivalent).
• Exposure to statistical inference or machine learning (e.g. STAT 460 / 560 or MATH 419 or CS 540 or equivalent).
• If you are not sure, come talk to me after one or two lectures.

Feedback

To me

This is the first time I teach an online graduate course. Feel free to email me if something does not work for you (keeping in mind thought that there are several constraints so I may not always be able accommodate all requests)

To you

• Evaluation scheme:
  • Short exercises (10%):
    • every 1-2 weeks, short problems to encourage you to do the readings and stay on top of the material.
    • I willl drop the lowest grade. Grades will be essentially checking that the work was done.
    • I will experiment with peer evaluation so that you can still get feedback on correctness of the answer.
  • Participation (10%): for each interval [Tuesday, following Tuesday), ask OR answer ONE question on Piazza.
    • Enroll to Piazza today: https://piazza.com/ubc.ca/winterterm22020/stat520a
    • If your name on Piazza is different than the one in the class list, send me a email with your student ID and Piazza username/email
    • You can skip one week of your choice without penalty
  • Final project (80%): more information will be posted soon
• Note: I will include polls during the lectures. These are NOT graded so that people in different time zones are not penalized. BUT: answer the best you can, so that I can adjust the pace of the lectures.
• For new graduate students, please keep in mind that graduate courses are very different than undergraduate course. Heavy shift away from summative assessment. Example: look at the documents that faculty positions request to submit.

Getting help

• There is an extended Q&A session starting at the end of each lecture
• By appointment
• Piazza (I will redirect substantive email questions to Piazza)

References

• Christian Robert, The Bayesian Choice, 2nd edition.
  • Available for free inside UBC VPN: https://tinyurl.com/y5sz278w
  • Excellent coverage of theoretical aspects
• Andrew Gelman et al., Bayesian Data Analysis, 3rd edition
  • Available for free from the author’s website: http://www.stat.columbia.edu/~gelman/book/BDA3.pdf
  • Encyclopedic
  • Many fascinating applied case studies and underlying applied best practices
• Richard McElreath, Statistical Rethinking
  • Available for free inside UBC VPN: https://tinyurl.com/y44mjrqu
  • Very pedagogic and gentle
  • Code examples: http://xcelab.net/rm/statistical-rethinking/
  • See also lively lectures from the author, linked from: https://github.com/rmcelreath/statrethinking_winter2019
• Resources on Monte Carlo:
  • Monte Carlo theory, methods and examples (2013). Art Owen.
    • Available from the author’s webpage..
    • Excellent treatment of Importance Sampling in particular.
  • Monte Carlo strategies in scientific computing (2001). Jun S. Liu.
    • Available through SpringerLink behind UBC paywalls or VPN.
    • Good reference for the basics but missing important new developments from the past 15 years.
  • Handbook of Monte Carlo Methods (2011). Dirk P. Kroese, Thomas Taimre, and Zdravko I. Botev.
    • Seems to cover some slightly more recent algorithms but still missing lots of developments that happened in this decade.
• Resources on probabilistic programming
  • Blang, developed in my group, which efficiently supports both discrete, combinatorial and continuous random variables; especially useful for hard non-convex problems.
  • Probabilistic programming for hacker, Cameron Davidson-Pilon
    • Probabilistic programming book giving a programmer-centric view of Bayesian analysis
    • Many lively examples
    • Free ebook: http://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/
  • Frameworks focussing on autodiff+HMC, e.g. for continuous random variable and nice log-concave targets: Stan, Edward, pyMC3, pyro (some support discrete variable at various degrees of efficiency).
  • Languages focussing on maximum expressivity: Anglican, Birch