The Broken Science Initiative 2025
Recently, I had the opportunity to visit Scottsdale, Arizona, to participate in the Broken Science Initiative (BSI) founded in 2022 by Greg Glassman and Emily Kaplan. The BSI's mission is to address systemic issues within modern science, particularly in fields like medicine and social sciences. Greg, with his mathematical mind and classical education, co-created the world-famous CrossFit movement, which eventually sold for millions.
For the most part, science is indeed broken. From the top, most people see the perceived arrogance of scientists, the bias and misconduct in research, the sensationalism in the media, and the politicization of science. Simply read the endless articles of image manipulation, fabrication, falsification, and plagiarism by individuals and organizations. Obvious cases include Merck’s Vioxx and Purdue’s OxyContin scandals. Both resulted in thousands of deaths. Another example, the statistics surrounding the COVID-19 tests: we lacked sensitivity or specificity for those tests, yet we were mandating bold claims and decisions, all without knowing what the tests did.
Then, there are discussions about whether statin medications affect mortality or whether high cholesterol causes atherosclerosis. The debates around whether Lucky Charms are ‘healthier’ than eggs, processed plant-based meats over red meat, low-fat vs. full-fat dairy, and sports drinks for everyone are scenarios that reflect how marketing, misinterpretations of guidelines, and lobbyists skew public perceptions, leading to controversial health claims.
More seriously, less than half of cancer studies are reproducible, and over 90% of nutrition studies are not reproducible. This means that if the researchers repeat the experiments, they will get a different result. Certainly cancer research is complex, and the reasons for irreproducibility are many. This fact signifies that the models they used for prediction are inaccurate. That’s not good science, nor beneficial for anyone.
Dana-Farber, the preeminent Harvard cancer research institution, is accused of falsifying images in more than 53 published works. Other topics concern whether chemotherapy cures cancer, are breast mammograms beneficial, evidence-based medicine, whether vaccines work, or why we dump millions of pounds of pesticides on our foods, pretending that it doesn’t adversely affect health. Or the belief that chopping off healthy body parts from gender dysphoric kids, actually decreases suicide mortality, statistics affects all of these issues.
Research has devolved into paper mills, scientific chop-shops, publish or peril culture, only publishing positive results, and ghostwriting. Indeed, not all science is broken, and upstanding people are doing good research. But the water’s less clear where there’s vast riches. When asked what’s wrong with research, my answer is money and more money. US healthcare spending is about to reach $5 trillion, approximately $15 thousand per person. Hospital care and prescription drugs are industries worth $1.5 billion and $450 billion, respectively.
Despite all this spending, in the US, we’re faced with the fact that over 40% of adults are considered obese and/or diabetic, and a significant percentage of individuals aged 65 and over cannot get up off the floor with one or two hands. Or that cardiovascular disease kills over half a million people annually, or that suicide kills fifty thousand people annually.
I arrived at BSI Central, and Greg Glassman started the class by saying, “Let’s start with the truth,” namely that modern science is the source and repository of man’s objective knowledge.
The truth is the opposite of a lie. In research, scientists must be committed to finding the truth, even if finding the truth means that you won’t be published or receive a Nobel Prize. The truth is believing that it will result in the best outcome that could possibly happen, no matter what. You have to believe that the truth will set you free. If scientists believed the truth had no bearing on human flourishing, the whole enterprise would be pointless. Einstein was an office clerk who revolutionized science and never used statistics or randomized controlled trials (RCTs), or p-values. All he was after was the truth.
When researchers don’t aim for the truth, they spend years justifying their lies, eventually destroying themselves. The Dana-Farber Institute research tampering, the Alzheimer's research scandals, and the plagiarism case involving former defamed Harvard President Claudine Gay are recent examples.
We’re surrounded by fellow scientists and physicians who seem perfectly content with the status quo. They dare not question the research that drives the guidelines, forcing this checkbox-type medicine under the guise of evidence-based medicine we all face today. Career scientists operate by the principles that will maximally benefit themselves or their families economically and this pollutes the entire scientific enterprise. Even worse, when an entire society lies, it becomes an authoritarian state, a tyranny, and mayhem results. If you need references to believe this, just read Solzhenitsyn’s Gulag Archipelago, the Japanese stories of Unit 731, or the Nazi research experiments in Auschwitz. You will never be able to “unknow” these atrocities once read. Voltaire once said, “If you see fraud, and you don’t say fraud, you are a fraud.” You can’t just sit back when you see things going off the rails. It leads to us giving up our fundamental freedoms and a totalitarian society.
Glassman highlights that all scientific knowledge is siloed in models. A model maps a fact to a future unrealized fact known as a prediction. One highlighted problem is that consensus has taken over the truthfulness of predictions. Predictive power is the sole source of reliability and provides the basis for any rational trust in science.
The BSI argues that modern science is “broken” due to its lack of predictive value, over-reliance on consensus rather than empirical validation, and reproducibility issues. They critique traditional statistical methods like p-values and confidence intervals, suggesting these contribute to “broken science.”
Some of my favorite quotes from the BSI include:
“When consensus replaces predictive value, science becomes nonsense.”
“Science, by definition, is the unique field that predicts measurable event which come to pass when tested.”
“When an article gets around to saying “most scientists believe…”, it is time to go back to the comics section.”
“The goal of science is to predict, implying that the process is under observation.”
“Science requires models to make nontrivial predictions, not necessarily the ultimate prediction.”
“The theory of relativity didn’t invalidate Newton’s laws; it just required the domain to be tightened.”
“The acid test of science is not the number, or the quality of papers published, but in scientific models validated.
“Science is embodied in models of the real world that have demonstrated predictive power.”
“Science can be neither theological nor atheistic.”
“The church was and is the most significant purveyor of misinformation in history.”
“RCTs have only been around for 75 years.”
The BSI offers educational resources and community-driven approaches like The Medical Society and The Journal Club for healthcare providers. These initiatives aim to foster a network where members can critically analyze existing literature and engage in discussions about scientific methodology and outcomes, thereby contributing significantly to the BSI's mission. Consider joining these groups.
At its base, the BSI is about how we use statistics in science. The BSI is Greg’s and Emily’s wish for the world. The Broken Science Initiative brings people together to talk about how science is truly broken and fundamental ways to fix it. Some of the scientists that have attended: Epidemiologist Jay Bhattacharya, cardiologist Aseem Malhotra, astrophysicists Anton Garrett and Paul Coles, regenerative farmer Joel Salatin, cardiologist Malcolm McKendrick, cancer researcher Thomas Seyfried, author Gary Taubes, nutritionist Zoe Harcombe, Professor Gerd Ginzenger, and many others.
The BSI highlights scientific misconduct and fraud at academic institutions, government agencies, and the private sector. The BSI’s mission is to change the fundamentals about how we think about the statistics, predictive models, and data in science.
Refs
https://www.theguardian.com/commentisfree/2023/aug/09/scientific-misconduct-retraction-watch
https://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/308269/
Ioannidis, J. P. A. (2005). "Why Most Published Research Findings Are False." PLOS Medicine, 2(8), e12
Peter Rodgers, Andy Collings (2021) Reproducibility in Cancer Biology: What have we learned? eLife 10:e75830
https://www.nature.com/articles/d41586-021-03691-0
Thanks for coming! So glad you enjoyed the weekend!