# On Health Evidence and Policy

Tweet Follow @hazula

An article in the New York Times reminds us that, especially with the way people consume information today, a scientific, evidence-based recommendation is not sufficient to change a policy. A well-constructed PR campaign is also required.

See: Mammogram Math regarding the recent brouhaha that resulted from a recommendation to reduce mammogram screening frequency in not-at-risk populations.

A description of a commonly misunderstood probability relevant in this case:

The exact weight the panel gave to these considerations is unclear, but one factor that was clearly relevant was the problem of frequent false positives when testing for a relatively rare condition. A little vignette with made-up numbers may shed some light. Assume there is a screening test for a certain cancer that is 95 percent accurate; that is, if someone has the cancer, the test will be positive 95 percent of the time. Let’s also assume that if someone doesn’t have the cancer, the test will be positive just 1 percent of the time. Assume further that 0.5 percent — one out of 200 people — actually have this type of cancer. Now imagine that you’ve taken the test and that your doctor somberly intones that you’ve tested positive. Does this mean you’re likely to have the cancer? Surprisingly, the answer is no.

To see why, let’s suppose 100,000 screenings for this cancer are conducted. Of these, how many are positive? On average, 500 of these 100,000 people (0.5 percent of 100,000) will have cancer, and so, since 95 percent of these 500 people will test positive, we will have, on average, 475 positive tests (.95 x 500). Of the 99,500 people without cancer, 1 percent will test positive for a total of 995 false-positive tests (.01 x 99,500 = 995). Thus of the total of 1,470 positive tests (995 + 475 = 1,470), most of them (995) will be false positives, and so the probability of having this cancer given that you tested positive for it is only 475/1,470, or about 32 percent! This is to be contrasted with the probability that you will test positive given that you have the cancer, which by assumption is 95 percent.

The arithmetic may be trivial, but the answer is decidedly counterintuitive and hence easy to reject or ignore. Most people don’t naturally think probabilistically, nor do they respond appropriately to very large or very small numbers. For many, the only probability values they know are “50-50” and “one in a million.”

blog comments powered by Disqus