It has become clear that much of medical research is flawed, and so inherently biased that much of it/most of it simply cannot be relied upon. One of the strongest critics of this current situation is a brilliant statistician, Professor John P Ionnadis. His seminal paper on the subject of medical research, which is nearly ten years old now, was entitled ‘Why Most Published Research Findings Are False ‘. I include the abstract here:
‘There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias1.’
Has his work been contradicted by anyone? The answer would be a resounding… no. In fact, all that has happened over the last ten years is more and more confirmation that medical research has become worse.
This is an incredibly worrying situation, yet very few people seem in the slightest bothered. The status quo remains in status. When new medical studies come out the press continue to regurgitate the findings as though they are unquestioned gospel. Experts have maintained their status as demi-gods, to be fawned upon as though their work is beyond any possible reproach.
Guidelines, the ones that instruct doctors on how to treat various conditions, are still published without any provisos. Guidelines which are based on evidence that… ‘may often be simply accurate measures of the prevailing bias.’ But woe betide any doctor that fails to follow said guidelines, for they may well be struck off the medical register. In the US, you could end up in jail.
All of these things are bad enough, and there are many other problems. However, in this blog, I want to focus on another issue. Namely, what about placebo controlled studies? Just to make it clear, for those who know a great deal about this area, I am not looking at the issue of ‘what the hell is in placebos anyway, cos it sure as hell ain’t inert substances.’ Whilst the fact that you cannot find out what manufacturers actually put in placebos, which should be inert ‘sugar pills’, but most certainly are not, is extremely important, that is an issue for another day.
Today’s issue is as follows. We have reached a situation in medical research where it may never be possible to find out if certain treatments actually work. Sub-header… ‘And in which case we are all doomed.’
Here is the context. Once a treatment has been found to be superior to a placebo, it will be deemed unethical ever to carry out a placebo controlled study ever again. That may not mean much to many people, so I shall expand – using a concrete example (yes, statins again).
If placebo controlled studies have shown that statins reduce the risk of heart disease, and for the sake of argument let us accept that this is true, where does this leave us? It leaves us in the position whereby, if anyone wanted to set up a study to try and disprove that statins are no better than placebo, they will never be given permission to do so.
Why not? Well, before you are allowed to carry out a clinical study, you have to present it to an ethics committee. This committee will look at the proposal and decide if it is indeed ‘ethical.’ Exactly what this means is up for debate. However, if you decided to study the speed at which cars need to run into children, to result in a fifty per cent mortality rate, I imagine you would be turned down by the ethics committee.
More prosaically, if you have found that statins reduce the risk of dying of heart disease vs placebo, then you will no longer be allowed to do a placebo controlled statin trial ever again. The reason for this is that you have already ‘proved’ that statins are superior to placebo. So it will argued that any volunteer placed in the placebo arm of your study would be suffering avoidable harm. Bong! Ethics committee says no. We know statins work, so it is unethical not to give them.
The only studies the ethics committees will allow would be statins vs. statins and a new drug. Equally you would not be allowed to study a new drug vs. placebo, at least not for an indication where statins had shown a benefit. Because everyone ‘at risk’ should be on a statin already.
Now, I have some sympathy for pharmaceutical companies in this situation. If statins reduced the risk of heart disease by 50% (made up figure), then any new drug can only provide an incremental benefit over statins – there is only 50% possible benefit left. So you need to study more people, over a longer period, to demonstrate superiority over statins. A higher hurdle than statins had to get over to be approved.
In another way, obviously, I have less sympathy. Let us suggest that all of the statin trials were biased. Let us further suggest that statins do not have any benefit over placebo. Is there any evidence for this? Well, the only major non pharmaceutical funded study on statins vs placebo was ALLHAT-LLP. Which was run by the National Institutes for Health (NIH). It was reported thus:
‘Washington, DC – Surprising results of an unblinded but randomized comparison of pravastatin (Pravachol® – Bristol-Myers Squibb) vs “usual care” in patients with hypertension and moderate hypercholesterolemia enrolled in the Antihypertensive and Lipid Lowering Treatment to Prevent Heart Attack Trial (ALLHAT-LLT) show that pravastatin did not significantly reduce either all-cause mortality or fatal or nonfatal coronary heart disease (CHD) in these patients.’
So, no benefit at all. This study was immediately attacked by all the ‘experts’ and dismissed as being useless, not enough LDL lowering, not enough difference from standard care blah, blah. Nothing to see here, move along.
However, I find it interesting that the only statin study which was not funded by the pharmaceutical industry was completely negative. You may even believe that this would give people pause for thought. If so, silly you.
Where does this leave us though? Well, as already stated, you can never, ever, do another statin vs placebo study. For it would be unethical to do so. You can never do a cholesterol lowering study on any other drug vs placebo either, for it would be unethical to do so. If the statin trials were all correct and unbiased and without the slightest doubt attached to them….fine. If, however, these trials were simply accurate measures of the prevailing bias then we are completely screwed.
This leaves us in a situation whereby if we test other drugs against statins, we are testing a drug against a drug that we cannot be certain has any benefits at all. So, what can we prove? Nothing. Which means that the very foundations of all future research in this area have been built on a bog.
So, what can we do? Carry on believing that all the research done is correct and above any suspicion of bias and manipulation. If so, fine, but you may have trouble sleeping at night. If not, you are going to have to tear apart all of the research that has been done, and do it again. I think that makes the task of Hercules look pretty easy.