Research claims published in medical journals can turn out to be exaggerated, misleading and wrong
I USUALLY avoid reading health supplements in newspapers as they are full of contradictory guff and puff about what is supposed to be good or bad for you.
The puff emanates from drug companies exaggerating the supposed benefits of their products and the guff from PR companies promoting cosmetics or “food as medicine” style products. Governments and health agencies get in on the act too and over-emphasise certain research results to promote public health.
I remember being pregnant and getting annoyed by the alarmist guidelines issued to expectant mothers regarding foetal and newborn health. The risks of some behaviours seemed to be deliberately exaggerated because health agencies believe that nuance is dangerous.
For instance, a few glasses of wine a week for a pregnant woman is quite harmless, but public health policy, particularly in the US, warns against any alcohol intake in pregnancy. Or you will be warned that you could smother your baby accidentally if you let it sleep in your bed. The evidence shows that where that has happened, it was most likely when the mother’s drunk boyfriend fell asleep with the baby on the couch. But to tackle the behaviour of the stupid, the responsible are made pay the price. This excessively cautionary principle applies across a wide range of behaviours.
But public health policy is just one of many filters between truth and what you end up reading in the papers. The challenge when you are reading an article about the supposed effect of a pill, vitamin or lifestyle behaviour is to figure out how many other filters there are. You might be alert to the PR company ruses, or politicians making political capital out of a health story. You will put your sceptic hat on if some alternative medicine practitioner makes renewed claims about homeopathy. You can double-check with Ben Goldacre’s bad science site (badscience.net) where he debunks an astonishing amount of rubbish that purports to be based on empirical evidence.
But really, the easiest thing to do is ignore all the silly articles. I’m usually happy to take my doctor’s advice. However, two recent articles did bother me because they warn your doctor could be wrong too. Both articles were about the content of medical journals – the ones doctors read to keep up on the latest research.
One study, which was referenced by Goldacre, concerned advertising in those journals and the other, covered in the November edition of the Atlantic magazine, was about the research papers published in the journals. Both studies focused on well-known and highly respected publications such as the Lancet, the Journal of the American Medical Association and the New England Journal of Medicine.
Researchers in the Netherlands examined all advertisements in the journals between 2003 and 2005 and selected ads that made specific claims about the effect of a drug based on the results of drug trials. They checked the references and found that only 39 per cent of these adverts referenced a high-quality trial that supported their claim. The Dutch study is just one in a series that exposed the dubious claims of advertising by drug companies.
But the story on the academic papers published in the journals was much more worrying. It was about the work of Dr John Ioannidis, a Greek doctor who specialises in researching research. The Atlanticconcluded that "he and his team have shown, again and again, and in many different ways, that much of what biomedical researchers conclude in published studies – conclusions that doctors keep in mind when they prescribe antibiotics or blood-pressure medication, or when they advise us to consume more fibre or less meat, or when they recommend surgery for heart disease or back pain – is misleading, exaggerated, and often flat-out wrong".
Part of the problem is that medical journals will publish the most eye-catching research and so researchers desperate to get published will always be looking for a supposed breakthrough, which in reality is just a fluke.
In Ioannidis’s most damning paper, he showed that the results of between one-third and half of the 49 most cited pieces of research in the medical world were later contradicted or shown to have lesser effects than initially claimed. The papers included studies many of us are familiar with. For instance a claim that hormone replacement therapy reduces the risk of heart disease has been soundly contradicted. So have studies that claim vitamin E significantly reduces the risk of heart disease or that stents are superior to balloon angioplasty in managing heart disease. The problem is that often the better quality research that refutes the initial claim is not as popular as the original paper.
Ioannidis’s work is extremely popular and is widely accepted in the medical community, but he acknowledges this acceptance hasn’t changed how research is conducted. Bad practice is still widespread throughout medicine. The bad research continues because most of the time it doesn’t matter. A good doctor will use judgment, instinct and choices to treat you. You just have to hope their choices are reasonably well informed and accept that in science being wrong is an occasional side effect.