Book Review of 5 Applied Bayesian Statistics Books

post by Jan Christian Refsgaard (jan-christian-refsgaard) · 2021-05-21T10:23:51.672Z · LW · GW · 8 comments

Contents

  TLDR
  Short review of each Book
  Recommendations / Extra considerations
    Math:
  My Experience with the books
None
8 comments

There are 5 strong contender for best Bayesian Statistics book

TLDR

• Statistical Rethinking, henceforth SR^[1]
  • Up to speed fast, no integrals, very intuitive approach.
• Doing Bayesian Data Analysis, henceforth The Dog Book^[2]
  • This is the easiest book. If your goal is only to create simple models and you aren't interested in understanding the details, then this is the book for you.
• A Student’s Guide to Bayesian Statistics, henceforth Student's Guide^[3]
  • This book has the opposite focus of the Dog book. Here the author slowly goes through the philosophy of Bayes with an intuitive mathematical approach.
• Regression and Other Stories, henceforth ROS^[4]
  • Good you want a slower and thorough approach where you also learn the Frequentest perspective.
• Bayesian Data Analysis, henceforth BDA^[5]
  • The most advanced text, very math heavy, best as a second book after reading one or two of the others, unless you are already a statistician.

Irregardless of which one you pick, watching the YouTube lectures for either SR or Student's Guide is very helpful

Short review of each Book

The Dog Book: Is the easiest book is thus if you do not strive for understanding but simply want to quickly get to a skill level where you need to develop a not to fancy model then this is a great book, It is also a good reference book as each chapter is based on a specific link and function and regression variable type, thus, if you want to do an Bayesian ANOVA you simply look for chapters named something like "categorical predictor with metric outcome".

Student's Guide: This book has the opposite focus as the Dog book. Here the author slowly goes trough the philosophy of bayes with a mathematical intuitive approach. It looks like a very good reference book. A Bayesian professor has recommended it as one of the best introductions to STAN. Chapter 8 is also very good. It starts with a graph of the relationship between all likelihoods and then goes trough EACH and EVERY ONE with an intuitive example and some nice plots. I would recommend everyone to read the chapter and/or to use the chapter as a reference whenever you have a few 'candidate' likelihoods in your head. Reading Chapter 12 in SR will subsequently teach you to create mixtures of these likelihoods if you need further hacks such as zero inflation. Remember the YouTube series explain the math very well. So use them as a supplement!

ROS: This book is a 'normal' statistics book written by Bayesians, thus it teaches both philosophies and have very great intuitive mathematical examples. It is a trophy of very intuitive considerations about model building, such as: $N(25,10)$ is statistically significantly different from 0, but hardly even 1 $\sigma$ away from from $N(10,10)$ because variances are additive, so the uncertainty of the 15 difference is $\sqrt{(}{10}^2+{10}^2)=14$. The same is true for interaction terms as they have two sources of error, thus we should a priori expect those to have wider posteriors! The slow approach makes it immensely readable for people like me 'who already know this', as half of the book is basically 'there be dragons' explanations of everything that can go wrong when you are doing a regression, and them doing the analysis twice to show the difference between the different philosophies.

SR: This is another great book, and it uses a level of math that is easier than ROS and Student's Guide but more rigorous than The Dog Book. This means that you get up to speed much faster, and it has you building quite advanced models by skipping the math and by heavily developing your intuition. Until Chapter 11 it's very great, but after than it starts introducing advanced concepts, and the material in Chapter 14-16 is not covered in the 4 books above. So it can also be bought as an "advanced supplement" to any of the 4 books above. Also the lectures are phenomenal and track well with the chapters, so I advise watching them prior to reading a chapter to get a big picture overview before going deeper.

BDA: This book used to be the bible. It's very mathy compared to the 4 other and seems to be written by field experts. Part 1 and 2 seem 'coherent' and are actually quite good for understanding the math that the other books use but don't explain well.. Part 3 and 4 are mathy versions of what are superficially covered in the above 4 books and most of Part 5 is simply state of the art Bayesian modelling expressed using only math. I think it might be slightly more intuitive than reading the source papers - but only slightly.

Recommendations / Extra considerations

Causal Inference: The books SR and ROS put extra emphases on causality, thus if you have observational data, where you want to predict the outcome of an intervention these books are 'extra' good. SR emphasizes Judea Pearl's graph based approach which is superior when doing fancy models, given they are both introductory I think that ROS actually teaches you to guard against more causal errors, so it's hard to declare a winner.

Math:

Dog Book << SR < ROS < Studet's Guide << BDA

• I have taken less than 15 ECTS of math (studied biology or sociology), then pick The Dog Book or SR.
• I have taken some math (Engineering or Econometrics): Student's Guide or ROS
• I have an undergrad degree in math: BDA

I want to make cool models before page 200:

• SR or Dog Book

I am a patient learner:

• ROS or Student's Guide

I want a good reference book, all are decent, SR is worst because it has 'playful' chapter titles such as "Ulysses’ Compass" and "The Golem of Prague" which actually serves as helpful memetic when you are reading, but becomes a pain a year later when you need to look up things.

My Experience with the books

I have not read all the books from cover to cover, here is my experience with each one:

• Doing Bayesian Data Analysis (The Dog Book)
  • Read cover to cover
• Statistical Rethinking (SR)
  • Read Chapter 12 and half of 14
    • Mixture Likelihoods and Covariance Models.
  • Watched all 20 lectures
  • Solved assignments via study groups
• Regression and Other Stories (ROS)
  • Read Chapter 1-9
  • Read a lot of Andrew Gelmans Blog
• Bayesian Data Analysis (BDA)
  • Read Chapter 1-13
• A Student’s Guide to Bayesian Statistics (Student's Guide)
  • Skimmed the earlier chapters.
  • Read Chapter 8
  • Watched about 10-20 hours of his YouTube lectures

---

1. Richard McElreath "Statistical Rethinking" ↩︎

2. John Kruschke "Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan ↩︎

3. Ben Lambert "A Student’s Guide to Bayesian Statistics" ↩︎

4. Gelman, Hill and Vehtari, “Regression and Other Stories” ↩︎

5. Andrew Gelman, John Carlin, Hal Stern, David Dunson, Aki Vehtari, and Donald Rubin. "Bayesian Data Analysis" ↩︎

8 comments

Comments sorted by top scores.