I'm Starting To Get It

After considerably more reading, this is the way I'm understanding it. Let me know what I'm missing or getting wrong.

The key to Bayes’ theorem is that it makes explicit the relationship between two events, and the probability of one event given that the other event has occurred.

Here’s a simple example: Say you want to know if it rained last night, so you go outside and touch the grass to see if it’s wet. Before you touch the grass there is some probability that it rained during the night, maybe based on a forecast of 40% chance of rain. But after you touch the grass and feel that it is wet, there is a new (higher) probability that it rained. Bayes’ theorem gives you a way to calculate the new probability that it rained last night, given the evidence of wet grass.

You can’t conclude that it rained last night based on the fact that the grass is wet, because the sprinklers may have come on or it may just be dew. But if you know something about the relationship between the event of the grass being wet and the event of it having rained, then you can calculate how likely it is that rain is the cause of the wet grass (CORRECTION - that reference to 'cause' is objectionable, we're only discussing correlation). The real insight of Bayes’ theorem is that the likelihood that rain is the cause of the wet grass is related to the probability of the grass being wet when it hasn’t rained.

If you happen to know that there is a very low probability of the grass being wet in the morning after a night with no rain (maybe you don’t have sprinklers, and you live in a dry climate with very little dew), then wet grass is a strong indicator of rain. But if there is a high probability of the grass being wet on a morning after no rain, then wet grass is a very weak indicator of rain.

This is fairly intuitive, and so maybe it doesn’t seem very revolutionary. However, what Bayes’ theorem does is it makes this kind of reasoning explicit and calculable. Bayes’ theorem justifies this kind of reasoning by formally spelling out how and why it works. An understanding of Bayes’ theorem also helps one avoid mistakes of probabilistic reasoning, e.g. thinking that wet grass is a stronger indicator of rain than it really is. 

A Call to (Bayesian) Missionaries

Math is Tough

I need someone to show me the light.

I've tried, but I just can't seem to understand the far-reaching implications of Bayes' theorem. The more people I ask for help, the more I become convinced that 'Bayesian' is a synonym for 'inarticulate math-head'.

See, I want to understand why it matters. Why it's supposedly transformational.

Here's the example I have heard most frequently used to explain Bayes' theroem: Imagine there is a test to screen for a particular kind of cancer. When the test is administered to people who don't have the cancer, 99.5% of those screened will be correctly flagged as cancer-free (low false positive rate). When the test is administered to people who DO have the cancer, 99.9% of those screened will be correctly flagged as having the cancer (low false negative rate). In the general population, .01% of people have this cancer, so that would be about 30,000 people in the US!

Two questions: 1) Assuming the test is fairly cheap to administer, should this screening be done routinely for all people? 2) If I am flagged positive for having the cancer, what is the probability that I actually have the cancer?

The answer to question 2 is surprising, and that's why this example is used to promote Bayes' theorem. Using Bayes' theorem (I'll spare you the math) you find that if you're flagged as having the cancer, there is only about a 2% CORRECTION 16.7% chance that you actually have it. So the answer to question 1 is probably no, because getting a positive result still means you have a very low chance of actually having the cancer (instead the screening should probably only be done for people at high risk).

That seems like a pretty good example because it's easy to relate to and has a non-intuitive outcome. However, I think it must actually be a very poor example of the power of Bayes' theorem, because I can very easily work that same problem out with just standard probabilistic reasoning and get the same answer! You just divide the number of genuine positives (people who have the cancer who test positive for the cancer) by the number of total positives (false positives plus genuine positives) to get the likelihood that any particular positive result is a genuine positive result (the 2% result comes from the fact that the false positive rate is high relative to how rare the cancer is). It's simple, easy, and doesn't require Bayes' theorem.

But there are a lot of people who are excited about Bayes and his famous theorem, so there must be more to it than this example. So can someone please help me to understand?