Of all the things you should bear in mind when looking at health stories, this is probably the single most important. Association does not mean causation. The reason why this is so important is that studies that have only found associations make up the vast bulk of scare stories in the media:
Here is a typical recent headline, which you may have seen:
Eating red meat regularly ‘dramatically increases the risk of death from heart disease’
It is true that this newspaper headline does not actually state that eating red meat causes heart disease. Not quite, but very nearly, and you could be forgiven for thinking that it does. Read it again, and you will not see the word cause anywhere. It is just very implied very strongly
However, as you get into the article itself, any distinction between association and causation fades almost to nothing:
‘Senior author Professor Frank Hu, from Harvard School of Public Health in Boston, US, said: ‘This study provides clear evidence that regular consumption of red meat, especially processed meat, contributes substantially to premature death.
‘On the other hand, choosing more healthful sources of protein in place of red meat can confer significant health benefits by reducing chronic disease morbidity (illness) and mortality.’
The study found that cutting red meat out of the diet led to significant benefits. Replacing one serving of red meat with an equivalent serving of fish reduced mortality risk by 7 per cent.’
At this point we are heading into the territory of Bill Clinton in his impeachment trial where the meaning of words it being stretched to their very limit. ‘But what is, is?’
I defy anyone to read those paragraphs and not conclude the following:
1: These researchers proved that eating red meat causes premature death
2: The researchers further proved that cutting out red meat and replacing with fish reduced mortality risk by 7 per cent.
I don’t think you could be blamed for thinking these two things. Because that appears to be exactly what was said. Or was it? Were you just being fooled by a complex conjuring trick made up of carefully chosen words designed to bewilder.
Here are the actual conclusions of the paper:
‘Red meat consumption is associated with an increased risk of total, CVD, and cancer mortality.’
Note the word associated. Where is the word cause? It isn’t there, because this study could never, ever, prove causality. Why not? Because it was an observational study (actually it was a review of two other observational studies).
In an observational study you do not do anything active. You just study that things that people do, or eat, and see if any associations emerge. When you find an association the next question you have to ask is the following. Are you looking at yellow fingers, or smoking.
It is certainly true that yellow fingers are associated with a higher rate of heart disease Does it follow that yellow fingers cause heart disease? No, of course not, what it means is that people with yellow fingers are usually people who smoke. And smoking vastly increases the risk of dying of heart disease.
In this case the distinction between the cause and the association is blatantly obvious – or at it has become so after fifty years of research made it clear Indeed, if I were now to try and claim that having yellow fingers causes heart disease, you would look at me as though I were an idiot – and I would be.
Yet, when a study finds that eating red meat is associated with a higher risk of heart disease, we seem to rush headlong into the conclusion that eating red meat consumption almost certainly causes heart disease. But red meat could well just be the equivalent of yellow fingers.
You think not? In that case you are probably thinking that red meat contains saturated fat, and saturated fat raises cholesterol levels, and raised cholesterol levels cause heart disease. If you played this little causal chain in your mind, you would most certainly not be alone in doing so.
It is something that our brains seem hard-wired to do…
‘….our brains and nervous systems constitute a belief-generating machine, an engine that produces beliefs without any particular respect for what is real or true and what is not. This belief engine selects information from the environment, shapes it, combines it with information from memory, and produces beliefs that are generally consistent with beliefs already held. This system is as capable of generating fallacious beliefs as it is of generating beliefs that are in line with truth.’ http://www.csicop.org/SI/show/belief_engine/
We cannot seem to help ourselves from linking things together to create causal chains, or beliefs, that certain things cause other things to happen. This is emotional, it is exceedingly powerful, and deconstructing such beliefs is the work of Hercules.
In this particular case, though, you would be hard pressed to use this belief as an explanation. How do I know this? I know that because the Harvard team found that those who at the most red meat actually had the lowest cholesterol levels. This table (figures taken from the paper itself) divides people into five groups/quintiles. Those in quintile 1 ate the least red meat, those in quintile 5 the most.
Total Red Meat Intake Quintile, Servings per Day
% with high
Pan A; Sun Qi;, ScD; Bernstein A; et al: ‘Red Meat Consumption and Mortality:Results From 2 Prospective Cohort Studies.’ Arch Intern. Med.Published online March 12, 2012.doi:10.1001/archinternmed.2011.228
The authors chose not to make any comment at all on this finding. Although you might have thought it worth a quick mention. Had they found rising cholesterol levels with increased meat consumption you can be absolutely certain they would have presented this as a clear cut causal chain. So how did eating read meat cause an increase in the rate of heart disease? Because it just did….through some mechanism unknown to medical science? The evil power of redness.
What is far more relevant is that they also found that those who ate the most red meat also smoked the most, exercised the least, ate far more calories in total, and were more likely to have diabetes. But it was the red meat that killed them from heart disease….you think? Even if red med included pork, and unprocessed red meat included hamburgers.
This study demonstrated, as if any further demonstration were required, that a whole bunch of unhealthy lifestyle factors: smoking, taking no exercise, drinking, eating fast food are all linked together. But I think we knew this already.
Hi Dr. Kendrick,
First, the differences are not statistically significant. Second, if they are looking at cholestrol levels, they would have to do a multivariate analysis to make sure it is adjusted for confounding variables. You cannot look at raw data and come to conclusions.
You would be mislead if you looked at the raw death rates in the same data too.
Good deconstruction but It is important to be precise about the scientific method. Correct is “Association does not necessarily mean causation,” or “Association does not mean causation unless the association is strong and the underlying mechanism makes sense. This is how things are understood in physical sciences. In some sense, there is only association. God does not whisper in our ear that the electron is charged. We make an association between an electron source and response of a detector. As I always say, all of astronomy is observational but you can wait for a returning space capsule within a hundred yards of where it lands.
You do do something active when you make an association. Associations test hypotheses (however unstated). Pan, et al. were testing the hypothesis that red met increases mortality. The hypothesis failed because the association was so weak, the underlying principle was mostly that Walter Willett finds it disgusting and the associated observations, which you indicated, compromise the original hypothesis. Association is a strong indicator of smoking as a cause of lung disease because the odds ratio is 22, not 1.3 and because we have an underlying mechanism in tissue damage and enzyme inhibition. Remember, you cannot prove a scientific theory, you can only disprove one. Along those lines, a lack of association as in fat vs. CVD, as you and others have well shown, is a strong indication of a lack of causation. The emphasis on prospective study may have to do with self absorption of (some) MDs. An association that you were not looking for is free from experimenter bias. Science does not run on arbitrary rules. Pan, et al. is bad because it’s dishonest. This is not ad hominem; I am sure that Pan is an honorable man. They are all honorable men, but the study does not present an honest analysis of the data as yours above. I think we can only show up these studies for what they are using the correct principles.
Thanks for your thoughtful comment. I would not fully agree that associations can prove causation (I am a Popperian). Of course, if the association is exceedingly strong e.g. smoking and lung cancer, you get as close to proof as is possible through observational studies. Your general points are, however, quite correct.
I think we are in basic agreement. Popper would probably say that you could not prove causation at all, only lack of causation. In any case, it is not true that “The authors chose not to make any comment at all on this finding [about cholesterol].” They actually say:
“”saturated fat and cholesterol from red meat may partially explained this association. The association between red meat and CVD mortality was moderately attenuated after further adjustment for saturated fat and cholesterol, suggesting a mediating role for these nutrients.” So, cholesterol had a causal effect because it didn’t have an effect. You probably missed this because of your elitist insistence on logic and common sense. What planet is this?
One thing that these newspaper interpretations of results never mention is what they’re basing their percentages on, and if they’re a relative or absolute risk.
On an absolute factor: Say if out of 100 people, 1 will get cancer, a 7% increase would mean 8 people will get cancer.
On a relative factor: If out of 100 people, 1 will get cancer, a 7% increase would mean 1.07 people will get cancer.
Only problem is, by detailing WHAT the increase is based on, i.e. relative or absolute, it can dilute how significant the findings are.
Saying 1.07 compared to 1 person will get cancer and most people would not even bat an eyelid. But the human mind sees an absolute factor primarily.
Hence why I don’t listen to crap on the news any more.
I fully agree. I am writing another book at the moment and it includes quite a lot about the mis-use of statistics in general, with particular reference to absolute risk/benefit. Somewhat frighteningly I have to report that no doctor I have yet asked has been able to accurately explain to me what the difference is between relative and absolute risk. I am sure that there must be some out there who do, but not many.