Better in their words…

Yesterday we began the introduction to this series. Today, the end of the beginning.

Seven deadly sins of churnalism
As we thought about these posts, we concentrated on explaining the errors researchers and journalists commonly make in the hope of preparing the reader to recognize these errors. We identified seven errors that underlie most healthcare churnalism. In jest, we started calling them the seven sins of churnalism. (As much as we tried, we could not match the churnalistic sins to lust, gluttony, greed, sloth, wrath, envy and pride). Over the course of these posts, we will link back to these sins as we identify them in the articles we discuss and we will expand and clarify their meanings. We introduce them here to prepare you for the deep dive to follow.

Sin #1  Observational studies almost never prove causation

You know that correlation does not guarantee causation. Just because people who carry lighters are more likely to develop lung cancer does not mean that carrying a lighter causes lung cancer. However, we regularly see researchers stumble across an association and conclude that there is a causative relationship. It is easy to find these relationships, and it is an important component of medical research — some associations actually do turn out to be causative. However, we consider it a sin when the researcher assumes causation and then recommends an intervention based on that presumption. Smokers should stop carrying lighters to reduce their risk of lung cancer.

Sin #2  Extrapolation and generalization

We always want more and there is no place this is more evident than in churnalism. We want the drug that works in mice to work in humans. We want the results of an exercise experiment in four male college students to apply to a 65 year old retired waitress. Researchers want their studies to have broad, far-reaching significance. Readers want breakthroughs or surprising discoveries. Unfortunately, because breakthroughs are rare, researchers and journalism hype minor findings through extrapolation and generalization, another sin of churnalism.

Sin #3  Ignoring confounding, selection bias, and other epidemiological errors.

It will take us some time to fully explain this tricky sin. There are a number of biases in epidemiological research and ignoring them is another churnalistic sin. When researchers or journalists commit this sin they tell a story in a way that is not supported by the data. The irony of this sin is that in committing it, the sinner often misses a more interesting analysis of the data.

Sin #4  Neglecting plausibility

At the heart of so much of churnalism is ignoring our inner BS detector. So many stories make utterly implausible claims; claims that if a friend made them we would laugh, “get out of here.” We will harp on articles that propose that something we eat — carrots or blueberries or nuts—are the most effective cancer treatment or most deadly toxin that has ever existed. These reports tend to fill health news reports. Abandoning common sense is a deadly sin of churnalism.

Sin #5  The Disclaim and Pivot maneuver.

The disclaim and pivot maneuver is a red flag that the article you are reading is churnalism. The author inserts a disclaimer but then immediately turns back to the flawed reasoning that underlies the article:

“Of course this finding might only represent association and not causation but…”

“You should ask your doctor before changing your diet but the research suggests…”

“Future studies are needed to confirm this finding but in the meantime you might consider…”

These disclaimers are true — you should ask your doctor and we would benefit from more studies — but authors frequently use this as a cop-out, a thin excuse for lazy reporting. The maneuver allows the author to nod to the reasonable while asserting the far-fetched.

Sin #6  Keep testing; report just once

A classic error of churnalism is not appreciating how many times a question was asked before it was answered. Imagine, we want to judge how well you shoot free throws. We let you shoot 10 times and count the shots that sink. That is a fair way to measure your skill. Now imagine we give you a video camera and let you record yourself shooting and allow you to submit footage of 10 shots. Obviously, with enough time and patience, you can make yourself look like Jose Calderon during the 2008-2009 season just by showing us your best ten consecutive shots. What does this have to do with biomedicine? Researchers have tremendous flexibility in conducting medical science and make choices, sometimes consciously, sometimes not, about what to report. There has been some terrifically creative research that makes it clear that if you keep testing, but report just once, you can show just about anything.

Sin #7  Being incurious

The last, but perhaps most important, sin of churnalism is a lack of curiosity. (This one, at least, we can pair with sloth). Journalists do not ask the researcher the tough questions. They do not seek out alternate points of view. They do not spend the time to figure out that although the study does not show what it purports to, what they are breathlessly reporting, the research does tell us that there is something here worth discussing. A rich and lively controversy. An important, yet unanswerable question.

Our Examples

The new stories that we’ll mention, from the mundane to the absurd, will be something we stumbled across. We will not search out stories; we will let stories find us. We expect to find some in our own daily reading and listening. We expect to notice some on Twitter. We often get articles forwarded to us by friends or colleagues who know about our obsession with the interpretation of medical research and know that a particular outlandish claim would really irritate us. We are happy if you, the reader, send articles our way.

Controversy

It is likely that some of what we will write will be controversial and some of the researchers and journalists whose work we cover will probably disagree with us. We are not shy about expressing our opinions. We will always cite data and references to support our statements. If you disagree with a point we hope you will give us a chance to convince you and will take the time to follow our reasoning and evidence. Science and medicine is always evolving and we fully expect that some of what we say will need to be refined or adjusted over time. (Our readers have not been shy about pointing out examples of medical reversals we used in Ending Medical Reversal that turned out to not be medical reversals). Life is a long lesson that teaches you to keep an open mind. Readers who disagree should comment or tweet @ us. We are always happy to engage in a conversation.

Onward

Our use of the scientific method to better our health is a great (perhaps the greatest)  accomplishment of our species. It has led to truly amazing advances in our health, well-being, and longevity. This research, and its resulting effect on society, should be celebrated and communicated to the broadest audience. Good health journalism can not only educate but motivate a new generation of thinkers, researchers, and leaders. Churnalism, however, can have the opposite effect. It can confuse and frustrate readers and lead people to make poor decisions about their healthcare. It can further the public distrust of science and undermine the enthusiasm of the next generation of investigators.

We are looking forward to taking a deep look at the origins of bad health and science reporting. We’ll explore problems with stories about the research into what we eat, how we exercise, how we sleep, and the things we do for fun or in pursuit of health. We even look into how the media has covered a particularly troublesome new virus.

We hope that these posts will help equip consumers of the news with the tools to separate the intriguing, promising, and marvelous from the hyped, uninformed, and harmful. In doing so, we hope to inspire people to do good, productive, and important science, to practice medicine well, and to communicate the advances in science and healthcare in reliable, interesting, and informative ways.