Category Archives: Math/Statistics

Study of the Week: Choices, oh my, the choices

Source: Study of the Week: Choices, oh my, the choices

I used to think *the* data yielded *the* result. This paper challenged that notion.

Aug 15 Comment Share

Summer tends to be slow for medical evidence so this week I will dip into the slide deck of my favorite studies. I will tell you about one of the most shocking studies I have seen in my career. It forever changed my view of evidence interpretation. Buckle up.

Medical studies aim to answer a question. This may be a trial comparing a drug to a placebo; it may be an observational study looking for a correlation between two things, such as fitness and longevity.

Researchers collect data. They put the data in a spreadsheet. Then they analyze it.

It turns out there are many ways to analyze data—not only in the statistical tests to sort out signal from noise but also in the choices of how to handle variables.

When you read a medical study, the researchers tell you how they analyzed the data. In the best quality studies this is set out before the experiment.

But…they analyze it in only one way.

Yet there are actually many ways to analyze data.

In 2018, the scientist Brian Nosek, from the University of Virginia, tested the idea that results of studies may hinge on subjective decisions made during the process of analyzing the data.

Pause there. How can that be?

You would think that the creativity in science would come in the question asked or the design of the experiment—not something as banal-seeming as analyzing results.

I called Nosek’s paper the most important study of the year.

Here is a quote:

For years, when I read a scientific paper, I’ve thought that the data yielded the published result.

Nosek and colleagues recruited 29 teams of expert researchers to analyze one data set to answer one simple question: were professional soccer referees more likely to give red cards to dark-skin-toned players than light-skin-toned players?

The 29 teams of researchers analyzed the same data in 29 different ways. The picture shows the different approaches. The names are super-complicated. The point is that they are different.

Now look what happens:

Two-thirds of the expert teams of data scientists detected a significant result and one-third found no statistical difference. Two teams of experts found results that were highly suggestive of implicit bias amongst referees.

This paper rocked me to my core. No paper I’ve ever read in medicine has ever said we will analyze data in multiple ways. They analyze data in one way.

But you can see how much difference it might make.

It turns out that this has been replicated; and co-founder of Sensible Medicine, Vinay Prasad, was a co-author.

This group selected a clinical scenario in cancer medicine, such as survival in prostate, breast, or lung cancer. There have been many non-randomized studies that have compared outcomes in patients given different treatment regimens in these scenarios.

They then ran more than 300,000 regression models–each representing a publishable study. In each of these models they varied the use of methods to reduce selection bias.

And…similar to Nosek, they also found that varying analytic choices, would generate different outcomes. In fact, their findings suggested that more rigorous choices to reduce bias reduced the likelihood of positive findings.


Here is the point.

Let’s say you read a study that finds a statistically significant association. Let’s then say the odds ratio is 1.20, which means the exposure of something may increase the odds of an effect by 20%.

The 95% confidence intervals are 1.02-1.40, roughly meaning that had the experiment been run an infinite number of times, the true result would fall within a 2% increase or 40% increase.

This is a statistically significant result. The researchers will make positive conclusions. Media may cover it as a positive study, potentially a breakthrough. Public or medical opinion may change.

Yet this result came from the researchers’ chosen analytic method. What if they had chosen a different way to approach the analysis, like the 29 research teams in Nosek’s study?

There might be different results…and different conclusions…and different policies.


My conclusions are simple:

There are obvious things in medicine. Fixing a hip fracture, antibiotics for infection, insulin for Type 1 diabetes.

But the vast majority of our knowledge is far less certain. This is okay. Normal even.

Yet I wonder…what would be the effect of open data? What if experimental data were available to independent scientists that could then analyze the results in ways they see fit?

Truly important findings would surely pass muster.

But if a result can be affected by the choice of how to analyze the data, well, then, perhaps we, as in everyone, ought to be a lot less rapturous about its results.

Like CommentComment ShareShare

If you liked this post from Sensible Medicine, why not share it?


What Netflix can teach Universities

By Vinay Prasad, May 17, 2022

A lesson in courage

During the COVID19 pandemic, I looked around for universities to host debates or roundtables on school closure, mask mandates, business closure, lockdown, the varied interpretations of the IFR— in other words: the biggest policy issues of our day. And what did I find?

A single debate for JAMA, a couple of videos from across the pond (BMJ), and a debate for Johns Hopkins hosted by the great humanitarian & thinker Stef Baral. What about Stanford? Nothing; Yale? Crickets. Princeton? Harvard? Zilch.

Why did the most prestigious universities abdicate the responsibility to host debates? And worse: why do they still abdicate it? There are no debates on boosting 5-11 year olds, vaccine mandates for college kids, or the evidence FDA should demand for a yearly COVID shot.

The answer is simple: University administrators are jellyfish (spineless), and they are scared that some fraction of their faculty, staff, or students will label some position as harmful. Ergo, they do not want to host a debate, lest some fraction of their body be offended or hurt by a “harmful” idea.

What does that mean? Our society further slides into the abyss, making bad policy choices, and universities forfeit their position to podcasts and videos, such as Plenary Session, which do push a range of COVID19 ideas and guests.

Enter Netflix. Netflix recently told its employees.

Not everyone will like—or agree with—everything on our service. While every title is different, we approach them based on the same set of principles: we support the artistic expression of the creators we choose to work with; we program for a diversity of audiences and tastes; and we let viewers decide what’s appropriate for them, versus having Netflix censor specific artists or voices.

As employees we support the principle that Netflix offers a diversity of stories, even if we find some titles counter to our own personal values. Depending on your role, you may need to work on titles you perceive to be harmful. If you’d find it hard to support our content breadth, Netflix may not be the best place for you.

This is exactly the memo that universities should be sending their own faculty, students and staff.

“I know some of you like school closure, and some of you think it is a bad idea. We are going to debate it here. If you’d find it hard to support holding open debates, Stanford may not be the best place for you.”

“I know some of you favor mandatory college boosters, and some of you think it is a bad idea. We are going to debate it here. If you’d find it hard to support holding open debates, Yale may not be the best place for you.”

“I know some of you think boosting a healthy 5 year old who just had omicron is a genius move, and some of you think that only a moron would do it. We are going to debate it here. If you’d find it hard to support holding open debates, Harvard may not be the best place for you.”

University administrators need to marshal the courage to tell their staff, faculty and students to shut up, and hear a range of opinions. So we can make progress as a people. And if they won’t, I have one more letter to send:

“I know some administrators fear confrontation and prefer to avoid making trouble. As such, they capitulate to a noisy group on campus. If this is you, being an administrator may not be the best job for you.”

Vinay Prasad’s Observations and Thoughts is going to be the best newsletter you read. Sign up today, so you don’t miss a post.


Anatomy of the Hong Kong Disaster 

Well, if you are still willing to follow, obey the ‘official narrative’ I’m not sure what to say… mrossol

Anatomy of the Hong Kong Disaster 

Anatomy of the Hong Kong Disaster 

Browstone Institute, By Ian Miller   March 19, 2022  

It’s hard to believe that after two years of government policies completely failing to prevent the spread of COVID, there hasn’t been universal acceptance that attempting to control a highly infectious respiratory virus is nearly impossible.

Nearly every country that was once praised for their “response” to COVID has seen their numbers rise dramatically over time. 

The mitigation commands — mask mandates, vaccine passports, mandatory vaccinations, lockdowns and lockdowns for the “unvaccinated” have all been disastrous blunders; hopeless flailing borne out of a desire to “do something” and to coerce desired behavior.

So it should come as no surprise that Hong Kong has joined the long list of jurisdictions to see their much praised policies collapse.

Equally unsurprising is that media and Twitter promoted experts have yet again ignored the ramifications of Hong Kong’s startling increases. 

Media reports have consistently attempted to credit masking and other interventions with stopping COVID, but Hong Kong provides a brilliant example of premature celebration.

Back in May 2020, Vox published an article with the unequivocal headline: “How masks helped Hong Kong control the coronavirus,” with the subtitle “New research shows that universal mask-wearing may help slow the spread of Covid-19.”

The article contains so many misguided assumptions about the spread of COVID that it’s stunning to see it hasn’t been retracted, but it helpfully summarizes how often experts have seemingly been making it up as they go during the pandemic:

If any city in the world was likely to experience the worst effects of the coronavirus, Hong Kong would have been a top candidate. The urban area is densely populated and heavily reliant on packed public-transit systems, and it has very few open spaces. Moreover, a high-speed train connects Hong Kong to Wuhan, China, where the coronavirus originated.

Hong Kong, it seemed, was doomed.

But almost as soon as the outbreak first began in the city, millions of residents started wearing masks in public. One local told the Los Angeles Times that the government didn’t have to say anything before 99 percent of the population put them on.

Experts now say widespread mask usage appears to be a major reason, perhaps even the primary one, why the city hasn’t been devastated by the disease.

“If not for universal masking once we depart from our home every day, plus hand hygiene, Hong Kong would be like Italy long ago,” K.Y. Yuen, a Hong Kong microbiologist advising the government, told the Wall Street Journal last month.

“99 percent of the population put them on.”

“Experts now say widespread mask usage appears to be a major reason, perhaps even the primary one, why the city hasn’t been devastated by the disease.”

“If not universal masking once we depart from our home every day…Hong Kong would be like Italy,” said an expert government advisor.

These quotes illustrate how experts and their partners in the media have operated during COVID — present unproven claims with zero evidence, repeat them as stated fact, and use their assertions to enforce mandates based on appealing to their own (generally incompetent) authority.

We may never learn the motivations behind the great mask wearing shift in early 2020, when experts disregarded years of carefully collected pre-pandemic planning by recommending universal masking, but this article does provides a helpful explanation.

Many of them believed in propaganda.

“Not wearing masks in Hong Kong is like not wearing pants”

It’s important to highlight this quote from the Vox article, not just for the absurdity and horrifying ramifications of treating face masks as pants, but to illustrate just how dedicated the residents of the city have been to universal masking.

Almost every indoor environment has enforced masking, with what we’re told is strict societal enforcement:

As the Journal also noted, some taxi cabs and shops won’t let people inside unless they wear a mask. Someone walking around the city without a mask on invites harsh looks from passersby and even verbal reprimands. Even the public address system on Hong Kong’s metro asks riders to wear masks at all times.

So surely, SURELY if any jurisdiction on earth could achieve permanent elimination of COVID with masking, it would be Hong Kong.

Mask wearing is as ubiquitous as wearing pants, we’ve been told. Those who do not comply were shamed and barred from life as far back as May 2020, when many US states hadn’t yet mandated one of the most useless policies in world history. 

Survey data has confirmed that mask usage has remained remarkably high and demonstrably consistent over time. 

So has it worked?

Well, not exactly.

In just a matter of weeks, cases in Hong Kong rose from a daily average of 1 per million to 5,089 per million, an increase of 508,800%.

Mask compliance remained unchanged. The Science™ was wrong. The Experts™ were wrong. Even with perhaps the world’s most dedicated mask wearing populace, the numbers have exploded.

And unfortunately, it’s not just cases, deaths have risen to startling new highs:

Despite Twitter promoted nutritionist Eric Feigl-Ding asserting that Hong Kong’s masking and interventions illustrated how to “defeat COVID,” newly reported deaths there are nearly 4x higher than they’ve ever been in the US.

Oh, and you may recall how the expert microbiologist advising the Hong Kong government claimed that without universal masking, Hong Kong would be like Italy…

Deaths in Hong Kong are now 2.5x higher than they were in Italy’s first wave.

Surprisingly, Vox recently tried to explain how the numbers in Hong Kong and other Asian countries could have risen so rapidly — yet if you search the article for “masks,” these are the only result: 

The United States is coming out of its deadliest phase of the pandemic and new Covid-19 cases are dropping sharply from their winter high. But more than 1,000 people in the US are still dying every day from the disease. Meanwhile, much of the country is relaxing requirements to wear face masks, testing is declining, and Covid-19 vaccination rates are leveling off. The US has not seen a wave of infections fueled by BA.2 just yet, but one may be looming.

How severe the next Covid-19 wave will be also hinges on how much the public is willing to take precautions, and many people are already putting masking and social distancing behind them. But as the pandemic has shown over and over, Covid-19 sets its own schedule.

Remarkable isn’t it? 

This same outlet said universal mask wearing stopped COVID in its tracks and that universal mask wearing was the key, the most important factor in limiting the spread of the virus. And in their article describing how Asia is failing to control COVID despite universal masking, they predictably call for MORE masking.

It’s just relentless, endless gaslighting and propaganda.

While many have blamed Hong Kong’s low vaccination rate for the recent stunning increase in deaths, a look at their vaccination dashboard shows that 82% of the population 12 and up has been fully vaccinated, and 91.4% have had at least one dose:

The age specific breakdowns might provide better clues, although the fact that they report 100.32% of ages 40-49 have had at least one dose does not inspire confidence on their data collection:

The lower rates among those ages 80 and above might be one of the more significant factors, although according to the Press Secretary of the United States, “we don’t know” if COVID affects elderly individuals like President Joe Biden more severely than other age groups:

It’s certainly possible that increased vaccination rates among the extremely elderly would have helped Hong Kong, yet we’ve repeatedly been told that death rates in the US have passed other countries because not enough people have been vaccinated. A quick glance at vaccinations in the US by age group shows stunningly high uptake among elderly populations.

And the focus on vaccinations ignores the fact that during previous surges elsewhere, Hong Kong has been specifically praised for controlling COVID with mask wearing. 

Why did that stop working? 

Why doesn’t anyone care?

Why is Anthony Fauci calling for reinstating mask mandates even with the US’s apparently superior vaccination rates?

No matter where you look, mask wearing hasn’t mattered.

Hong Kong is the latest, and perhaps most severe example of this failure. 

The mad rush to credit masking exemplifies the desire of media members and Twitter pundits to declare their allegiance to the correct set of ideological principles. The religious mantra of “believe in science” demands commitment to universal mask wearing, regardless of its unequivocal failure.

Hong Kong tried to control COVID with masking, and those same voices told us repeatedly that it was successful. Where were the retractions when then they subsequently reported one of the highest current death rates we’ve seen anywhere on earth?

While their cumulative rates remain lower than many European countries or the United States, COVID is not “over.” As we’ve heard repeatedly from Vox and the Twitter doctors who mindlessly maintain it’s “too soon” to lift mandates, the virus “sets its own schedule.”

Of course, they can’t mentally accept that it sets its own schedule, regardless of policy.



States Investigating Surge in Mortality Rate Among 18–49-Year-Olds, Majority Unrelated to COVID-19

Too early to say anything about COVID (the virus, the mandates, the lockdowns, etc.) is causal, but it is strange to me. mrossol

The Epoch Times, Jan 14 2022.   By Petr Svab

Health departments in several states confirmed to The Epoch Times that they are looking into a steep surge in the mortality rate for people aged 18 to 49 in 2021—a majority of which are not linked to COVID-19.

Deaths among people aged 18 to 49 increased more than 40 percent in the 12 months ending October 2021 compared to the same period in 2018–2019, before the pandemic, according to an analysis by The Epoch Times of death certificate data from the Centers for Disease Control and Prevention (CDC).

The agency doesn’t yet have full 2021 figures, as death certificate data has a lag of up to eight weeks or more.

The surge differed greatly from state to state, with the most dramatic increase in young-to-middle age deaths in the South, Midwest, and the West Coast, while the northeastern states generally saw much milder spikes. Public health authorities in several states with some of the largest increases are examining the issue.

Texas saw the 18 to 49 age mortality jump 61 percent, the second-highest increase in the country. Of that, less than 58 percent was attributed to COVID-19.


“Our Center of Health Statistics is looking at the data,” said Chris Van Deusen, the head of Media Relations at the Texas Department of State Health Services, via email. “We’ll get back with you.”

Florida, which saw an increase of 51 percent, 48 percent of that attributed to COVID-19, is also probing the matter.

Epoch Times Photo
A medical worker treats a non-COVID-19 patient in the ICU ward at UMass Memorial Medical Center in Worcester, Massachusetts on Jan. 4, 2022. (Joseph Prezioso/AFP via Getty Images)

“I am looking into it to see if there is some sort of correlation/causation,” said Jeremy Redfern, spokesman for the Florida Department of Health via email.

Nevada saw the highest increase, 65 percent, of which just 36 percent was attributed to COVID-19.

Shannon Litz, a public information officer at the Nevada Department of Health and Human Services, said via email she passed on questions regarding the mortality spike to the agency’s Office of Analytics “for review.”

The District of Columbia experienced an increase of 72 percent, none of it attributed to COVID-19.

Robert Mayfield, spokesman for D.C.’s health authority, referred The Epoch Times to the district’s Office of Chief Medical Examiner (OCME), which suggested it lacked the expertise to analyze the phenomenon.

“OCME does not currently have an epidemiologist (the position is being advertised) so it has no present ability to analyze the data,” said the office’s spokesman Rodney Adams via email.

Arizona recorded a 57 percent increase, 37 percent of which was attributed to COVID-19.

Arizona’s Department of Health Services couldn’t respond to questions regarding the issue because its data is “not yet finalized,” said Tom Herrmann, the agency’s public information officer, via email.

Other states with some of the highest increases were Tennessee (57 percent up, 33 percent attributed to COVID-19), California (55 percent up, 42 percent attributed to COVID-19), New Mexico (52 percent up, 33 percent attributed to COVID-19), and Louisiana (51 percent up, 32 percent attributed to COVID-19). None of their health authorities responded to requests for comment.

The mortality surge seemed to be significantly milder in the northeast. New Hampshire saw no increase, Massachusetts had only a 13 percent spike (24 percent of it attributed to COVID-19), and New York, one of the worst-hit by the pandemic in the region, was up 29 percent (30 percent of it attributed to COVID-19).

CDC data on the causes of those excess deaths aren’t yet available for 2021, aside from those involving COVID-19, pneumonia, and influenza. There were close to 6,000 excess pneumonia deaths that didn’t involve COVID-19 in the 18 to 49 age group in the 12 months ending October 2021. Influenza was only involved in 50 deaths in this age group, down from 550 in the same period pre-pandemic. The flu death count didn’t exclude those that also involved COVID-19 or pneumonia, the CDC noted.

Epoch Times Photo
Houston Fire Department paramedics transport a man suffering from breathing difficulties to a hospital on in Houston, Texas, on Sept. 14, 2021. (John Moore/Getty Images)

A part of the surge could be likely blamed on drug overdoses, which increased to more than 101,000 in the 12 months ending June 2021 from about 72,000 in 2019, the CDC estimated. About two-thirds of those deaths involved synthetic opioids such as fentanyl that are often smuggled to the United States from China via Mexico.

For those aged 50 to 84, mortality increased more than 27 percent, representing more than 470,000 excess deaths. Some 77 percent of the deaths had COVID-19 marked on the death certificate as the cause or a contributing factor.

For those 85 or older, mortality increased about 12 percent with more than 100,000 excess deaths. There were more than 130,000 COVID-related deaths in this group, indicating these seniors were less likely to die of a non-COVID-related cause from November 2020 to October 2021 than during the same period of 2018–2019.

Comparing 2020 to 2019, mortality increased some 24 percent for those aged 18 to 49, with less than a third of those excess deaths involving COVID-19. For those aged 50 to 84, mortality increased less than 20 percent, with over 70 percent of that involving COVID-19. For those even older, mortality jumped about 16 percent, with nearly 90 percent of it involving COVID-19.

For those under 18, mortality decreased about 0.4 percent in 2020 compared to 2019. In the 12 months ending October 2021, it fell some 3.3 percent compared to the same period in 2018–2019.




Petr Svab is a reporter covering New York. Previously, he covered national topics including politics, economy, education, and law enforcement.