Category Archives: Doctoring Data

Sunbathing is good for you

News announcer: ‘We interrupt the series of blogs on ‘what causes heart disease’ to bring you (slightly delayed), breaking news from Sweden… Sunbathing is good for you. Shock horror etc.’

Someone sent me this news story today, and I thought I should share it with you. For many, many, years I have been telling people that lying in the sun, getting a nice tan, is one of the healthiest things you can do. Despite the howls of anguish from all dermatologists telling us that one photon of sunlight is one photon too many. ‘You will cause people to die from skin cancer.’ Ho hum:

Why do sunbathers live longer than those who avoid the sun?

New research looks into the paradox that women who sunbathe are likely to live longer than those who avoid the sun, even though sunbathers are at an increased risk of developing skin cancer.

An analysis of information on 29,518 Swedish women who were followed for 20 years revealed that longer life expectancy among women with active sun exposure habits was related to a decrease in heart disease and noncancer/non-heart disease deaths, causing the relative contribution of death due to cancer to increase.

Whether the positive effect of sun exposure demonstrated in this observational study is mediated by vitamin D, another mechanism related to UV radiation, or by unmeasured bias cannot be determined. Therefore, additional research is warranted.

“We found smokers in the highest sun exposure group were at a similar risk as non-smokers avoiding sun exposure, indicating avoidance of sun exposure to be a risk factor of the same magnitude as smoking,” said Dr. Pelle Lindqvist, lead author of the Journal of Internal Medicine study. “Guidelines being too restrictive regarding sun exposure may do more harm than good for health.”1

There is a point here I think I should repeat… avoiding the sun is as risky for your overall health and life expectancy, as smoking. Which is pretty damned amazing? It has been estimated that smoking reduces life expectancy by six, on average. Thus, if you sunbathe regularly, it seems you can expect to live six years longer.

If I may indulge myself by quoting from my book ‘Doctoring Data’ on this very topic:

‘How about frightening people to stay out of the sun, or slap on factor 50 cream at the first suspicion that a deadly photon may sneak through 10 layers of protective clothing. Not necessarily a good idea, because without vitamin D synthesis in the skin, from exposure to the sun, there is significant danger that we can become vitamin D deficient, which can lead to all sort of other problems.

Here are just two stand-out facts from a major study in the Annals of Epidemiology entitled ‘Vitamin D for Cancer prevention.’

  • Women with higher solar UVB exposure had only half the incidence of breast cancer as those with lower solar exposure
  • Men with higher residential solar exposure had only half the incidence rate of fatal prostate cancer

To put that in simple English. If you spend longer in the sun, you may be far less likely to die of breast and prostate cancer. But what about the increased risk of dying of skin cancer! I have you cry. Well, what of it. Around 2,000 people a year die of malignant melanoma in the UK each year. It increased sun exposure were to double this figure we would have 2000 more cases.

On the other hand, breast cancer kills around 20,00 a year, as does prostate cancer. If we managed to halve the rate of breast and prostate cancer, we would reduce cancer deaths by 20,000 a year. Which is ten times as great as any potential increase in deaths from malignant melanoma.’

To what I wrote in Doctoring Data, I would further add that sun exposure is the best known way of increasing NO synthesis throughout the body. This protects the endothelium and, as you would expect, lowers blood pressure (the natural way). So, you are far less likely to die from CVD.

What this study highlights, once again (as with all advice on diet), what we are told to do by mainstream medical research, turns out to be actively damaging to health. Will advice on sun exposure now change? There is not the slightest, tiniest, possibility of this happening. Evidence has no impact on the pronouncements of the medical profession (at least not over the average human lifespan).

The only possible change I can see is that, whilst we will continue be hectored to stay out of the sun, at all possible costs, we will be advised to take vitamin D supplementation to make up for lack of sun exposure (even though there is little or no evidence that it actually does any good).

My advice is, and has always been. Sunshine is good for you. I have been saying this for twenty years. Ten years ago, whilst writing for Pulse Magazine in the UK I wrote an article called ‘Sunshine is good for you.’ I finished with the following:

Ponder this

I shall leave you to ponder the results of a study looking at people diagnosed with malignant melanomas, and then followed for five years.

‘Results: Sunburn, high intermittent sun exposure, skin awareness histories and solar elastosis were statistically significantly inversely associated with death from melanoma’

‘Conclusion: Sun exposure is associated with increased survival from melanoma.2

Did I say that sunshine is good for you? It even prevents malignant melanoma.

 

REFERENCES:

1: http://www.medicalnewstoday.com/releases/308202.php

The full study is: Avoidance of sun exposure as a risk factor for major causes of death: a competing risk analysis of the Melanoma in Southern Sweden cohort, P. G. Lindqvist, E. Epstein, K. Nielsen, M. Landin-Olsson, C. Ingvar and H. Olsson, Journal of Internal Medicine, doi: 10.1111/joim.12496, published online 16 March 2016.

2: Berwick M et al: Sun exposure and mortality from melanoma. J Natl Cancer Inst: 2005 Feb 2, 973(3):195-9

The Augean Stables – part II

It has become clear that much of medical research is flawed, and so inherently biased that much of it/most of it simply cannot be relied upon. One of the strongest critics of this current situation is a brilliant statistician, Professor John P Ionnadis. His seminal paper on the subject of medical research, which is nearly ten years old now, was entitled ‘Why Most Published Research Findings Are False ‘. I include the abstract here:

‘There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias1.’

Has his work been contradicted by anyone? The answer would be a resounding… no. In fact, all that has happened over the last ten years is more and more confirmation that medical research has become worse.

This is an incredibly worrying situation, yet very few people seem in the slightest bothered. The status quo remains in status. When new medical studies come out the press continue to regurgitate the findings as though they are unquestioned gospel. Experts have maintained their status as demi-gods, to be fawned upon as though their work is beyond any possible reproach.

Guidelines, the ones that instruct doctors on how to treat various conditions, are still published without any provisos. Guidelines which are based on evidence that… ‘may often be simply accurate measures of the prevailing bias.’ But woe betide any doctor that fails to follow said guidelines, for they may well be struck off the medical register. In the US, you could end up in jail.

All of these things are bad enough, and there are many other problems. However, in this blog, I want to focus on another issue. Namely, what about placebo controlled studies? Just to make it clear, for those who know a great deal about this area, I am not looking at the issue of ‘what the hell is in placebos anyway, cos it sure as hell ain’t inert substances.’ Whilst the fact that you cannot find out what manufacturers actually put in placebos, which should be inert ‘sugar pills’, but most certainly are not, is extremely important, that is an issue for another day.

Today’s issue is as follows. We have reached a situation in medical research where it may never be possible to find out if certain treatments actually work. Sub-header… ‘And in which case we are all doomed.

Here is the context. Once a treatment has been found to be superior to a placebo, it will be deemed unethical ever to carry out a placebo controlled study ever again. That may not mean much to many people, so I shall expand – using a concrete example (yes, statins again).

If placebo controlled studies have shown that statins reduce the risk of heart disease, and for the sake of argument let us accept that this is true, where does this leave us? It leaves us in the position whereby, if anyone wanted to set up a study to try and disprove that statins are no better than placebo, they will never be given permission to do so.

Why not? Well, before you are allowed to carry out a clinical study, you have to present it to an ethics committee. This committee will look at the proposal and decide if it is indeed ‘ethical.’ Exactly what this means is up for debate. However, if you decided to study the speed at which cars need to run into children, to result in a fifty per cent mortality rate, I imagine you would be turned down by the ethics committee.

More prosaically, if you have found that statins reduce the risk of dying of heart disease vs placebo, then you will no longer be allowed to do a placebo controlled statin trial ever again. The reason for this is that you have already ‘proved’ that statins are superior to placebo. So it will argued that any volunteer placed in the placebo arm of your study would be suffering avoidable harm. Bong! Ethics committee says no. We know statins work, so it is unethical not to give them.

The only studies the ethics committees will allow would be statins vs. statins and a new drug. Equally you would not be allowed to study a new drug vs. placebo, at least not for an indication where statins had shown a benefit. Because everyone ‘at risk’ should be on a statin already.

Now, I have some sympathy for pharmaceutical companies in this situation. If statins reduced the risk of heart disease by 50% (made up figure), then any new drug can only provide an incremental benefit over statins – there is only 50% possible benefit left. So you need to study more people, over a longer period, to demonstrate superiority over statins. A higher hurdle than statins had to get over to be approved.

In another way, obviously, I have less sympathy. Let us suggest that all of the statin trials were biased. Let us further suggest that statins do not have any benefit over placebo. Is there any evidence for this? Well, the only major non pharmaceutical funded study on statins vs placebo was ALLHAT-LLP. Which was run by the National Institutes for Health (NIH). It was reported thus:

‘Washington, DC – Surprising results of an unblinded but randomized comparison of pravastatin (Pravachol® – Bristol-Myers Squibb) vs “usual care” in patients with hypertension and moderate hypercholesterolemia enrolled in the Antihypertensive and Lipid Lowering Treatment to Prevent Heart Attack Trial (ALLHAT-LLT) show that pravastatin did not significantly reduce either all-cause mortality or fatal or nonfatal coronary heart disease (CHD) in these patients.’

So, no benefit at all. This study was immediately attacked by all the ‘experts’ and dismissed as being useless, not enough LDL lowering, not enough difference from standard care blah, blah. Nothing to see here, move along.

However, I find it interesting that the only statin study which was not funded by the pharmaceutical industry was completely negative. You may even believe that this would give people pause for thought. If so, silly you.

Where does this leave us though? Well, as already stated, you can never, ever, do another statin vs placebo study. For it would be unethical to do so. You can never do a cholesterol lowering study on any other drug vs placebo either, for it would be unethical to do so. If the statin trials were all correct and unbiased and without the slightest doubt attached to them….fine. If, however, these trials were simply accurate measures of the prevailing bias then we are completely screwed.

This leaves us in a situation whereby if we test other drugs against statins, we are testing a drug against a drug that we cannot be certain has any benefits at all. So, what can we prove? Nothing. Which means that the very foundations of all future research in this area have been built on a bog.

So, what can we do? Carry on believing that all the research done is correct and above any suspicion of bias and manipulation. If so, fine, but you may have trouble sleeping at night. If not, you are going to have to tear apart all of the research that has been done, and do it again. I think that makes the task of Hercules look pretty easy.

1: http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124

2: http://www.medscape.com/viewarticle/785851

Statins and cancer

(Ho hum, not again)

A number of people have written to me pointing out an outbreak of mass hysteria in the UK press about statins protecting against cancer. I suspect this hysteria has been repeated around the world. Here are the headlines from the eponymous Daily Mail

Statins slash risk of death by cancer: They slow tumour growth

by up to 50% reveal major studies

Experts say there is ‘overwhelming’ evidence that statins can treat cancer

Study showed they cut death rates for bone cancer patients by 55 per cent

GPs should make patients aware of pills’ new benefits, researchers say

I have been aware of claims that statins protect against cancer for many years. They pop up on a pretty regular basis. I have tended to ignore them on the basis that, anyone who is stupid enough to believe such research, deserves all the statins they can get.

However, such is the overblown hype this time, that I feel the need to rouse myself from my slumber, and explain why this is just complete rubbish. I don’t need to read the original studies to do this. I have read enough of these over the years. I hope this does not sound too arrogant, but I will happily apologise if any single thing I write here proves to be wrong.

Not randomised controlled studies

The studies quoted will not have been randomised and controlled. By which I mean they did not take, say, forty thousand people and split them into two, randomised, groups. One group to take statins the other to take a placebo. Then wait, say, five years to see what difference there was.

These studies will have been observational. By which I mean you look at people taking statins and see what happens to them vs. people who do not take statins. Such studies can show associations between two variables. But they cannot prove causality. (They cannot provide ‘overwhelming’ evidence of anything either). This is basic science, page one, paragraph one.

Just to provide one example of this. In 1987 a major observational study showed that women taking HRT had a more than forty per cent reduction in heart disease. At which point it was recommended that women took HRT to protect themselves against heart disease. This was, in fact, written into the guidelines of the American College of Physicians. To fail to prescribe HRT was considered medical malpractice in the USA.1

Some years later came the Women’s Health Initiative (WHI) study. The first randomised primary prevention trial to use HRT, and 17,000 women were involved.

‘Analysis of hazard ratios showed that after 5.2 years, there was a 29% increase in coronary heart disease risk, including an 18% risk of coronary heart disease mortality and a 32% increase risk of nonfatal myocardial infarction. There was a 20% increase in risk of fatal stroke and 50% increase in the risk of non fatal stroke in women assigned to HRT.2

So, a 42% reduction in heart disease turned into a 18% risk of dying of heart disease. In short, observational studies are hopelessly unreliable and often turn out to be complete nonsense. And there is a specific reason why I know these statins studies will be complete rubbish, which I will get to.

Relative not absolute risk

Once again, in these studies, we run into the distorting use of relative, not absolute risk. A fifty per cent reduction in risk can mean something, or nothing very much. It depends what the underlying risk was in the first place. In my book Doctoring Data I covered the use/misuse of relative risk in some depth.

Let us just say that if your underling risk of dying in the next five years is 50%, reducing that risk by 50% is a big deal. If the risk of dying in the next five years is 0.1%, then reducing that risk by 50% is five hundred times less of a big deal.

As for slowing tumour growth by 50%. Well, that could mean almost anything. Did you reduce tumour growth by 1%, 50% or some other number. And does reducing tumour growth actually reduce the risk of dying? Of course, you will always find some super rare cancer e.g. bone cancer, where death rates are cut by 55%.

I would imagine this meant about three deaths verses seven in bone cancer. Basically, however small the absolute figures can be to get to a relative risk reduction of 55%. I would guess there will be no statistical significance figure attached to this reduction. Many questions, almost none of them well be answered, you will find.

The elephant in the room (raised cholesterol protects against cancer)

Here, however, is the big issue. People with higher cholesterol levels are far less likely to die of cancer. Add this to the fact that people with higher cholesterol levels are far more likely be prescribed statins, and you start off with the most gigantic built in bias that it is possible to find.

In 1992 (before statins were being prescribed to more than a select few) a conference was held to look at low blood cholesterol and associations with mortality3. Going back this far in time is important. After this, statin prescribing makes it very difficult to disentangle those with naturally low, or high, cholesterol levels vs. those who were taking statins.

All the major studies of the time were reviewed, with nearly one million participants. As you can see from my little graph, reproduced from the figures in the paper, as cholesterol levels rise, the risk of cancer falls. For women, if your cholesterol level is below four, the risk of dying of cancer is 38% higher than if your cholesterol level is above 6.2mmol/l. In men we are looking at a 27% greater risk with low cholesterol levels. {See chart)

CL-vs-CR

Thus any observational study on lowering cholesterol with statins starts off with a massive inbuilt bias in the two populations. You are looking at one group of people who have a much lower risk of cancer to start with, then giving them statins, then declaring that statins protect against cancer….. just the most absolute unscientific codswallop.

As final warning. Be careful about lowering cholesterol too far. A very large Japanese study (that you will never have heard of, because it was not very supportive of statins) looked at prescribing statins to over forty seven thousand people over six years. As they found:

‘The patients with an exceptionally low TC (total cholesterol) concentration, the so-called ‘hyper-responders’ to simvastatin, had a higher relative risk of death from malignancy than in the other patient groups.’4

In fact, the rate of death from cancer in those whose cholesterol fell the most dramatically was increased by three hundred and thirty per cent (relative risk, apologies for doing this, but I do not know the absolute risk). The authors added this warning:

‘Further analysis is necessary to elucidate why the hyper-responders had an increased risk of death; their baseline characteristics will be described and discussed in detail in the future. Nevertheless, the health of patients who show a remarkable decrease in TC or LDL-C concentration with low-dose statin therapy should be monitored closely.’

Can I return to my slumbers on this issue now?

 

References

1: American College of Phyisicians. Guidelines for Counselling Post-Menopausal Women about Preventative Hormone Therapy. Ann Intern Med. 117:1038-41. (1992)

2: Writing group for the Women’s Health Initiative Investigators. ‘Risks and benefits of oestrogen plus progestin in healthy postmenopausal women. Principal results from the Women’s Health Initiative Randomized controlled Trials’ JAMA (2002)

3: Jabobs et al: Conference on Low Blood Cholesterol and Mortality: Circulation Vol 86, No 3 September 1992

4: Matsuzaki M et al: Large Scale Cohort Study of the Relationship Between Serum Cholesterol Concentration and Coronary Events With Low-Dose Simvastatin Therapy in Japanese Patients With Hypercholesterolemia Primary Prevention Cohort Study of the Japan Lipid Intervention Trial (J-LIT). Circ J 2002; 66: 1087 –1095

The dog that did not bark in the night

Some of you may have noticed this study, others may not. The amazing ‘wonderdrug’ trial proving that cholesterol lowering drugs have unparalleled benefits on preventing stroke. Here is just one headline from the Daily Express. A major newspaper in the UK.

Statins slash stroke risk by 30 per cent: Millions more should be given drug, say experts

New research has found that the wonderdrugs – which include statins and fibrates – can slash the risk of suffering a stroke by a third in the elderly. And experts now say there is clear evidence that even among the over-75s – a group not routinely prescribed statins – people can benefit from the life-saving drugs.

It is yet more evidence that the cholesterol-lowering drugs are lifesavers and that their benefits outweigh the potential side effects. Lead researcher Christophe Tzourio, Professor of Epidemiology at the University of Bordeaux and Inserm, said: “A one third reduction in stroke risk, if confirmed, could have an important effect on public health.”1

And so on and so forth.

Colleagues of mine love to wave articles like this at me with a triumphant smirk. ‘Seems you’re wrong about cholesterol lowering after all.’ What do you say to that? Eh..’ I usually ask them if they actually read the study. ‘Primary prevention with lipid lowering drugs and long term risk of vascular events in older people: population based cohort study.’2 I ask them this question, but I know that they’ve not. I find it rare to come across a doctor who would ever deign do such a thing as read a scientific paper.

However, when studies like this come out, I do feel the need to raise my enthusiasm to a sufficient level to have a peek at the paper. In this case it was rather easy. This paper was published in the British Medical Journal (BMJ), and I get it delivered to me every week by post. What a quaint thing, actual physical reading material.

My first problem, before I even started reading this study, is that I knew beforehand that a raised cholesterol level is not a risk factor for stroke. Never has been, not anywhere, not in any study I have read. Whilst you can find studies claiming that a raised cholesterol level (LDL) is a risk factor for heart disease [ and you can find others that show the opposite], I have yet to find any study demonstrating any association between raised cholesterol and stroke.

Here, for example, is a short extract from one massive study, the biggest, which looked at four hundred and fifty thousand people over seven million years of observation. It was published in the Lancet:

‘The associations of blood cholesterol and diastolic blood pressure with subsequent stroke rates were investigated by review of 45 prospective observational cohorts involving 450 000 individuals with 5-30 years of follow-up (mean 16 years, total 7·3 million person-years of observation), during which 13 397 participants were recorded as having had a stroke.

Most of these were fatal strokes in studies that recorded only mortality and not incidence, but about one-quarter were from studies that recorded both fatal and non-fatal strokes. After standardisation for age, there was no association between blood cholesterol and stroke except, perhaps, in those under 45 years of age when screened. This lack of association was not influenced by adjustment for sex, diastolic blood pressure, history of coronary heart disease, or ethnicity (Asian or non-Asian).3 [My bold].

Now, if you are unable to find an association between cholesterol levels and stroke in seven point three million years of observation then, you know what, it just ain’t there. In fact, I challenge anyone reading this blog to provide any evidence that cholesterol levels are associated with overall stroke risk. Gulp, that makes me hostage to fortune.

This is why stroke associations struggle when they talk about cholesterol and stroke. They seem desperate to say that raised cholesterol levels cause stroke, but just can’t. Here is how the National Stroke Association fudges the issue.

‘High cholesterol may raise your risk for stroke by increasing your risk for heart disease, a stroke risk factor.4

Whilst it is, of course, true that having heart disease does increase your risk of stroke, and vice-versa, the rest of this statement reveals a yawning gap in logic [For the sake of this argument, let us assume it is true that a raised cholesterol causes heart disease].

A (raised cholesterol) → B (heart disease) →C (Stroke)

A does not → C

Question. If A does not lead to C, how does A lead to B, then leading to C? I shall ask for this to become a question in the Oxford and Harvard entrance exams.

[BTW, if you can work this one out, then please feel free to let me know how it works. Exactly.]

Anyway. We find a study demonstrating that two cholesterol lowering drugs, in this case statins and fibrates, significantly reduce the risk of stroke. But a raised cholesterol level is not a risk factor for stroke. Which means that there can be no possibility that the benefit seen can have been due to cholesterol lowering? That, my friends, is simple logic. No need for Oxford and Harvard to get involved at all. This could be discussed on entrance to kindergarten.

Now, just to add to my short analysis this study I would like to draw your attention to something not remarked upon by the popular press at all. However, I thought that you may find it interesting. It was the following statement from the paper:

‘We found no association between lipid lowering drug use and coronary heart disease (hazard ratio 1.12, 0.90 to 1.40).’ [For those who hate figures/confidence intervals, sorry, I left them in for those who like them].

This was the dog that did not bark in the night.

In summary, here we have a study showing that cholesterol lowering reduced the risk of stroke, when a raised cholesterol level is not a risk factor for stroke. On the other hand, it failed to show any benefit on reducing the risk of heart disease. Some would consider that a study such as this raises more questions than answers. However, with wearisome inevitability, it has been twisted around to provide further proof that everyone should be taking statins. Sigh.

1:              http://www.express.co.uk/news/uk/578174/Statins-stroke-experts

2:              Alperovitch et al: BMJ 25 May 2015 pp12.

3:              Cholesterol, diastolic blood pressure, and stroke: 13 000 strokes in 450 000 people in 45 prospective cohorts The Lancet Volume 346, Issues 8991–8992, 30 December 1995, Pages 1647–1653

4:              http://www.stroke.org/stroke-resources/resource-library/cholesterol-and-stroke

Doctoring Data

I am pleased to announce that my book is finally written and edited and available to buy, on-line at http://doctoringdata.co.uk

The full, formal launch will not be until next year. But for those who cannot wait (hopefully several hundred million people), you can pre-order it now on a restricted print run. First come, first served as they say.

It has been a mighty effort to write, and I hope that people can both enjoy reading it, and feel that they have learned something by so doing. I am but your humble servant.

What is T?

Some questions puzzle me, and I search for the answer. For a number of years I am trying to establish. ‘What is T?’ My wife helpfully remarked that it is a drink with jam and bread. Ho, ho.

Moving swiftly on. My question relates to the concept of Number Needed to Treat (NNT). The NNT is a figure widely used in medicine as an outcome measure. It means how many people do you need to treat ‘T’ to achieve a benefit of some kind. The benefit can be many different things, for example: pain relief, curing a chest infection, improving pain and mobility following a hip replacement.

In these cases the ‘T’ is pretty clear cut. You have a medical problem and you intervene in some way to make it better, or cure it. But what is the ‘T’ when you are in the world of preventative medicine? If you are trying to stop something happening e.g. a heart attack, stroke, pulmonary embolism, or death, can you call preventing such things a form of ‘treatment?’

In reality, in preventative medicine, the ‘T’ turns into something else. It has become ‘P’, as in prevent. But treating and preventing are not the same thing, and you can’t use them interchangeably.

If you have a chest infection and I give you antibiotics then I have, in most cases, treated the infection. On the other hand, if you have a high blood pressure and I ‘treat’ it, all I have done is the lower the blood pressure. I have not immediately done anything else. A high blood pressure causes no symptoms, and there is nothing to be treated – other than future risk.

In fact, if lowering the blood pressure were a form of treatment, the NNT would be very nearly one, in that I will lower the blood pressure in almost every case where I prescribe a drug. But the NNT does not refer to the effect on blood pressure lowering; it refers to the number of people you need to treat to prevent, say, a stroke, by lowering the blood pressure.

As I hope is clear, in preventative medicine, the NNT should really be the NNP.

So what, you may think. Everyone working in this area knows that the NNT is really an NNP. You just need to know that when we use the term NNT, we are really talking about the number needed to treat to ‘prevent’ an event. Yes, this is true. However, the underlying problem with nomenclature does not disappear if we change NNT to NNP. The focus simply shifts to the word prevent itself. To prevent something means to stop it happening – forever.

Now, let us imagine death.

Can we prevent death? No, clearly we cannot. We do not make people immortal by lowering their blood pressure. All we can do, the very best we can possibly do, is to increase life expectancy – by some amount. Which means that prevention does not actually mean prevention. When we look at death as an outcome, prevention can only mean life extension. Or, turning this the other way round, the amount of time by which we delay something from happening.

At this point, I hope it has become clear that ‘T’ in preventative medicine has almost nothing to do with ‘treating.’ We treat nothing, we prevent nothing, we simply delay. At least that is all we can do with death. It is possible that we may prevent things such as non-fatal strokes, although we don’t really know, because we do not usually follow people up for long enough to be certain.

Why is this important? It is important for the following reason. When many clinical trials finish, and there is a difference in the number of deaths between the treatment and placebo arm, it is claimed that the difference represents lives that have ‘been saved.’ Which is another way of saying that death has been prevented which is, in turn, a different way of saying that death has been treated. NNT.

To give an example of how this work in real life I shall switch to statins and the Heart Protection Study (HPS)

Heart Protection Study
HPS1

This graph shows the ‘mortality’ curves for the statin and placebo arms. At the start of the trial everyone is alive, 100% in both groups. Five years later, the end of the study, 92.6% of those in the statin arm were still alive, and 90.8% of those in the placebo arm were still alive. A difference of 1.8%.

This was presented, in the HPS press-release, as follows:

‘In this trial, 10 thousand people were on a statin. If now, an extra 10 million high-risk people worldwide go onto statin treatment, this would save about 50,000 lives each year – that’s a thousand a week.’ http://www.ctsu.ox.ac.uk/~hps/pr.shtml
This is a very clear statement. Treat ten million people, and you will save 50,000 lives per week. But are these lives actually saved. No, of course not. Below, I have re-drawn the graph and extended both ‘survival’ lines by a year. We now have a year six.

HPS2

As I hope is clear, by year six, if we assume the lines continue along their previous trajectory, every single extra person who was alive in the statin arm, compared to the placebo arm, is now dead. Thus 1.8% of people did not have their lives ‘saved’. In fact, the average increase in survival time for these 1.8% was approximately six months. [Half of the 1.8% would have died after six months, which give you the mean/average].

So what is ‘T’ in this case. It is certainly not treatment, prevention, or number needed to treat to prevent death. Nor is it 1.8% of lives saved. It is a life extension of six months, for 1.8%.

Or, to put this another way. If you treat one hundred people at very high risk of heart disease (secondary prevention) with statins, what you are achieving is the following:

• 1.8 will live, on average, an extra 6 months.
• 98.2 will gain no benefit

What is ‘T?’ What indeed. Not perhaps what you first thought. T, at present, is taken to mean treatment. With preventative medicine treatment is taken to mean prevention, and prevention is taken to mean lives saved. But you cannot save a life, all you can do is extend life.

So, when someone says….

‘In this trial, 10 thousand people were on a statin. If now, an extra 10 million high-risk people worldwide go onto statin treatment, this would save about 50,000 lives each year – that’s a thousand a week.’

…they are talking nonsense.

In very short summary. NNT is a widely used treatment outcome, and it guides both clinical and economic decisions on what drugs should be used, or not used. It is a pity that in preventative medicine, NNT is meaningless, because ‘T’ has no value attached to it. Indeed, it might as well be a drink with jam and bread.

Can we believe any medical research – at all?

I have now finished my book, to be called ‘Doctoring Data.’ It has taken a long time to write, mainly because I had to bring together hundreds of different strands of thinking and research. Each strand seemed to get longer and longer as I attempted to pursue them to the end. In many cases I never really found the end.

Some ideas just keep stretching away forever and I had to give up, or else the book would have become a million pages long. And I was told three hundred and ten was to be my limit – or something like that. As if my genius could be contained to a mere hundred thousand words, or so.

Anyway, the main purpose of the book was to look at medical research and data, and try to make some sense of it for those who are interested in looking beyond a medical headline. The book was, at least in part, inspired by a paper written by John Ionnadis.

It was entitled ‘Why most published research findings are false.’ You can easily find it on the internet by searching the title. It is currently the most downloaded paper in recent medical scientific literature

The shortest summary of his paper is, as follows:

Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.’ J Ionnadis.

How can it be, you may think, that most published research is false? Surely research is the one area of human endeavour where bias and dogma are ruthlessly hunted down and destroyed. A scientific finding is a scientific finding….is it not? Did Francis Bacon die in vain?

As Dogbert might say. Hahahahahahahahahahaha!

Or you might be best to pay attention to the quote from Friedrich Nietzsche. ‘There are no facts, only interpretations.’ Of course, he was a bit bonkers, but there is an awful lot of truth to what he said. Especially in medical research. Facts are the most tricky little blighters to get hold of. Interpretation, however, that is stated as fact all over the place

Surely, though, we have ways to ensure that research is pure and objective, such as peer-review. A system of using respected ‘experts’ to check and approve papers before publication. This will weed out papers that are flawed, will it not. Well, here is what Richard Horton (editor of the Lancet) has to say on peer-review:

The mistake, of course, is to have thought that peer review was any more than a crude means of discovering the acceptability — not the validity — of a new finding. Editors and scientists alike insist on the pivotal importance of peer review. We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller. But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong.’

There we are, nice and reassuring to know that peer-review is such a fabulous system. As for the quality of published research itself, here is one of my favourite quotes by Drummond Rennie, at the time the Deputy Editor of the Journal of the American Medical Association.:

‘There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too selfserving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print.’

A view supported from a slightly different angle by Dr Marcia Agnell, who was the editor of the New England Journal of Medicine for two decades. This was, and remains, the single most powerful and influential medical journal in the world. At least it is, when it comes to citations and impact factor:

“It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine.” Dr Marcia Agnell

Here is a further view on the issue by Richard Smith, editor of the BMJ for many years. He wrote this in his blog:

Twenty years ago this week the statistician Doug Altman published an editorial in the BMJ arguing that much medical research was of poor quality and misleading. In his editorial entitled, “The Scandal of Poor Medical Research,” Altman wrote that much research was “seriously flawed through the use of inappropriate designs, unrepresentative samples, small samples, incorrect methods of analysis, and faulty interpretation.” Twenty years later I fear that things are not better but worse…

…The poor quality of much medical research is widely acknowledged,” wrote Altman, “yet disturbingly the leaders of the medical profession seem only minimally concerned about the problem and make no apparent efforts to find a solution.”

Altman’s conclusion was: “We need less research, better research, and research done for the right reasons. Abandoning using the number of publications as a measure of ability would be a start.”

Sadly, the BMJ could publish this editorial almost unchanged again this week. Small changes might be that ethics committees are now better equipped to detect scientific weakness and more journals employ statisticians. These quality assurance methods don’t, however, seem to be working as much of what is published continues to be misleading and of low quality. Indeed, we now understand that the problem doesn’t arise from amateurs dabbling in research but rather from career researchers.’

So, Ionnadis states that most research findings may often be simply accurate measures of the prevailing bias. Current and past editors of the three most respected and powerful medical journals in the world confirm that medical research is warped, biased and flawed and, in many cases simply not believable.

Would this be very evidence used by NICE* to tell us which drugs to use – for example. Why, yes it would be. So be afraid, be very afraid. For an idiotic politician (sorry for the tautology) recently made this announcement1.

‘A Labour government could reduce variation in access to drugs and procedures by making it mandatory for commissioners to follow national clinical guidelines, Andy Burnham has revealed.’ Andy Burnham was, at one time Secretary of State for Health. His is now shadow secretary of state for health. So, if the UK votes for Labour, it will mandatory for all doctors to follow the guidelines based on the evidence that comes from clinical trials.

Oh Joy.

*NICE stands for the National Institute for Care and Health Excellence. It is supposed to review all evidence for various healthcare areas and decree what is best practice. What NICE says tends to get taken up in many, many, other countries as their views are widely respected and acted upon.

1: http://www.hsj.co.uk/news/exclusive-labour-could-make-nice-guidance-mandatory/5076286.article