How to read a brain study: causality, reproducibility and hype
Peter Attia's journal clubs on epidemiology and statistics, Mark Gordon on research fraud, and placebo research on where effects stop all teach readers to question associations and small studies. It helps people judge health claims instead of chasing each headline.

The Short Version
A health headline is usually one study, and one study rarely shows what the headline claims. Across four conversations, the same habits keep coming up. First, ask whether an association could run backwards or be driven by something nobody measured. Then check how big the study was, what it was powered to find, and which control it was missing. Remember that published results can fail to replicate and are occasionally fabricated. And sort the effects expectation can produce from the ones it can't. Peter Attia and Andrew Huberman teach the statistics in their journal clubs [1][2]. The Alzheimer's fraud case Joe Rogan and Mark Gordon discuss shows why even prestigious papers need checking [3]. Huberman's placebo research shows why a controlled comparison matters, and where belief stops working [4]. None of this makes you cynical. It lets you decide which claims deserve a change in your behavior, and which deserve a shrug.
Table of contents
What They Said
Peter Attia: association is not causation, and the big question is always which way does the arrow point?
In their second journal club, Huberman presented a UK Biobank study of more than 85,000 people who wore wrist light sensors. People who got more daytime light and less nighttime light each had lower risk of major depression, PTSD, psychosis, self-harm and other psychiatric outcomes [1]. The findings are striking. More nighttime light went with rising risk of major depression, roughly 20 to 25 percent higher in the top quartiles. Self-harm stayed flat for three quartiles and then jumped about 30 percent in the top one. The top quartile of daytime light went with about a 20 percent reduction in major depression [1]. Attia used the paper to show how a careful reader handles numbers like these.
He starts with the mechanics. An odds ratio minus one gives the percent change, so 1.78 means 78 percent more risk and 0.7 means 30 percent less [1]. If a 95 percent confidence interval doesn't touch the line at one, the result is statistically significant [1]. He suggests drawing arrows on confusing figures so the direction is obvious [1]. Then he adds a warning that cuts against intuition. Huge studies can be "overpowered," so clinically irrelevant differences look significant. He said the varied error bar sizes in this paper reassured him [1]. A big sample is not automatically a trustworthy one, because size can make a trivial difference look important.
His biggest question was reverse causality. Depression itself may keep people indoors by day and awake at night [1]. He explained the idea with diet soda. An obese person may think, "How about instead of drinking a coke, I drink a diet coke," and so "the causality which you would impute to mean the drink is causing the obesity. It might be no. The obesity is causing the choice of drink" [1]. Applied to the light paper: "How much of the disruption in both day and night light exposure is the result of the depression. It's dysregulating the sleep" [1]. Then he states the limit plainly: "This is where epidemiology never allows us to determine this. And sadly these questions can only be answered through either direct randomization or Mendelian randomization" [1].
Attia didn't stop at doubt, though. He weighed the evidence. Asked how much of the effect was really due to light, he said he would go as high as 80/20. He cited the Bradford Hill criteria, especially the consistent dose-response and the strong biological plausibility [1]. He also noted that these light habits are a low lift compared with the exercise he asks of people [1]. Questioning an association doesn't mean throwing it out. It means asking how much weight it can carry, and how much acting on it would cost you if it turned out to be wrong.
He added one more reason observational data can mislead: the Hawthorne effect, where people change their behavior when they know they are being observed [1]. That matters for any study where participants wear a tracker. He also said sleep trackers and glucose monitors mostly teach lessons quickly by gamifying behavior. In his view, recovery scores predict performance poorly compared with morning resting heart rate [1].
Attia on metformin: how one analytic choice created a famous result
In the first journal club, Attia gave a full case study of a promising association falling apart [2]. The 2014 Bannister study drew on a UK registry of about 95,000 people. It compared type 2 diabetics taking only metformin with matched non-diabetics. The crude death rate was 14.4 per 1,000 person-years in the metformin group versus 15.2 in controls, and modeling showed a 15% relative reduction in all-cause mortality over a median of 2.8 years [2]. That was startling, because diabetes normally shortens life by about six years. It suggested metformin was doing something beyond lowering glucose [2]. The finding helped spark wide excitement about metformin as an anti-aging drug [2].
The catch, which Attia noticed only around 2018, was informative censoring. Metformin users who stopped the drug, were lost to follow-up or moved on to stronger medication were dropped from the analysis [2]. He compared it to a smoking study that stops counting the smokers who die: that group would look healthier than it really is [2]. The people most likely to do badly had been quietly removed from the metformin group.
A larger study by Keys and colleagues redid the analysis using a Danish registry of roughly half a million people. It compared 7,842 metformin users with the same number of matched non-diabetic singletons. It also compared 976 same-sex twin pairs in which only one twin had diabetes, which controls for genetics and upbringing [2]. The results reversed Bannister's. Crude deaths per 1,000 person-years were 24.93 for metformin users versus 16.86 for matched singletons, and 24.73 versus 12.94 among twins [2]. The hazard ratio was 1.48 for singletons and 1.32 to 1.33 after adjustment. For twins it was 2.15 unadjusted and 1.70 to 1.80 adjusted [2]. Applying Bannister-style censoring improved the numbers only modestly, to 1.39 for singletons and 1.97 for twins, nowhere near a survival advantage [2].
Attia was careful even here. Metformin failing to erase the disadvantage of diabetes doesn't prove it wasn't helping, since untreated diabetics might have done even worse. "absent a randomized control trial. We will never know the answer" [2]. He pointed to the TAME trial, led by Nir Barzilai, as the proper test in non-diabetics [2]. He also pointed to animal evidence from a rigorous setup. The NIH-funded Interventions Testing Program runs in parallel at three labs with genetically diverse mice, and metformin failed there while rapamycin repeatedly worked [2]. In the transcript he says "The itp is very rigorous," and explains that "it doesn't use an in-bred strain of mice It is done concurrently in three labs with very large sample sizing" [2]. That is replication built into the design.
He then applied the evidence to himself. He stopped metformin around 2018 after measuring a resting, fasted blood lactate of 1.6 millimoles. A healthy value would be roughly 0.3 to 0.6 [2]. He still prescribes it to insulin-resistant patients [2]. The lesson is that one bad study doesn't make a drug useless. It changes who the drug makes sense for.
Attia on the tools of epidemiology
The metformin discussion doubled as a crash course [2]:
- Baseline tables. Attia points readers first to Table 1. In the Keys study, one variable couldn't be matched: medications. The diabetics were far more likely to take lipid-lowering drugs (45.6% versus 15.4%), antiplatelet therapy (30% versus 14%) and blood pressure medication (about 63 to 65% versus 31%) [2]. Huberman joked that this is why he became an experimental scientist [2].
- Person-years. These normalize for different follow-up times. Five deaths per 1,000 person-years means five deaths whether you follow 1,000 people for a year or 500 people for two [2].
- Kaplan-Meier curves. They can only fall or stay flat, never rise [2]. Attia made the same point in the cancer paper: "Nobody comes back to life" [1].
- Statistical power. Before an experiment, you estimate the expected effect and calculate the sample size needed to detect it. A null result only rules out effects as large as the study was powered to find [2].
- Unmeasured confounders. These can never be fully removed. Studies find that people who don't eat meat live longer, but someone who makes that commitment probably also makes many other healthy choices that are hard to measure [2].
- Cost. Epidemiology makes up for its lack of randomization with larger samples, long follow-up and low cost. Randomized trials on tens of thousands of people are enormously expensive, and he cites the Women's Health Initiative as roughly a billion-dollar study [2].
- Supplements. Journals strictly limit length, so important data often sit in supplemental materials. Even the censoring analysis he described came from the Keys paper's supplement [2].
Attia on trials: endpoints, controls and what "better" means
The ipilimumab melanoma trial showed what a randomized trial can do that an association can't, and what it still asks of the reader [1]. The study enrolled a little under 700 patients with advanced metastatic melanoma. They were randomized 3:1:1 to ipilimumab plus a gp100 peptide vaccine, ipilimumab alone, or gp100 alone [1]. The control was a vaccine that had already failed. Attia called it "kind of a known placebo," and said the uneven allocation "increases statistical power" [1]. The primary endpoint was changed during the trial from response rate to overall survival. Attia considers overall survival the better measure [1].
He reads the survival curves for more than the median. Median survival was 6.4 months with gp100 alone versus about 10 months with ipilimumab. What mattered more was the tail: almost everyone in the control group had died by about 44 months, while around 20 percent in the treatment groups were alive at about 56 months [1]. He contrasted that with drugs approved for tiny median gains. One pancreatic cancer drug, he says, extended median survival by nine days at a cost of about $40,000 [1]. He also flagged a subgroup finding: women didn't clearly benefit, a pattern seen in about half of such studies. He wondered whether weight-based dosing meant men simply got more drug [1]. Reading a trial well means asking which outcome it measured, how large the gain really is, and who it did or didn't help.
Huberman: read the paper, not the headline, and ask whether the conclusions follow
Huberman's contribution runs in two directions. First, he criticized the title of his own chosen paper: "I think the title of this paper is terrible" [1]. The title says day and night light exposure are "associated with psychiatric disorders," which sounds alarming. "If this is where if this is where a newspaper headline, you'd be like, oh my goodness, well, what are you supposed to do?" [1]. The actual conclusion is that daytime light is protective and nighttime light is harmful [1]. Headlines, and sometimes titles, can point readers the wrong way.
Second, he offers a method. In a course at UC San Diego that grew from 50 students to more than 400, he taught four questions [2]. What question are they asking, broadly and specifically? What approach did they use? What did they find? And the one that separated the best students: do the conclusions follow? In his words: "You have to go back at the end and ask whether or not the conclusions the major conclusions drawn in the paper are Really substantiated by what they found and what they did" [2]. He warns that figure headers show "what they want you to think they found" [2]. He reads the title and abstract, then the figures, then returns to the text [2]. He is honest that "It's a skill that you develop over time" [2]. Early in that episode he framed the whole exercise as scientists looking at research "with a fresh eye asking does this paper really show what it claims to show Or not and in some cases the answer is yes, and in other cases the answer is no" [2]. He also corrected a common myth: overlapping error bars don't automatically mean there is no significant difference. That depends on the experiment and the statistics used [2].
He practices what he teaches. He noted that a much-cited study claiming light behind the knee could shift circadian rhythms was retracted [1]. When he recommended a light device, he said he has no relationship with the company and that its data are still early and unpublished [1]. When asked how much of the light effect was causal, he said "I think 65 to 75% of the effects are likely due to light directly," and admitted his bias from working on these cells since graduate school [1]. Stating your own bias is part of reading evidence honestly.
Mark Gordon and Joe Rogan: fraud, replication and the incentives behind them
On The Joe Rogan Experience, a discussion of Alzheimer's led to a major research fraud case [3]. According to the Science reporting they read on air, an investigation found that many studies by neuroscientist Eliezer Masliah contained "apparently falsified Western blots" and reused micrographs. A 300-page dossier flagged suspect images across 132 papers from 1997 to 2023 [3]. Rogan read the investigators' conclusion aloud: "In our opinion, this pattern of anomalous data raises credible concern for research, misconduct, and calls into question a remarkably large body of scientific work" [3]. Rogan's summary: "So he was stacking the deck in his favor" [3]. The stakes were large. Masliah helped lead the neuroscience division of the National Institute on Aging, with a budget of about $2.6 billion [3].
Gordon widened the point to reproducibility in general. "Reproducibility is where a researcher does a paper, makes a claim about the results of his science, and then people look at that and they want to go and reproduce it to prove it. They found that 70% of them can't be reproduced" [3]. He added that when the original scientists try to reproduce their own work, there is a "7% failure rate" [3]. He blamed publish-or-perish pressure to produce positive results for funding [3]. Rogan's verdict: "But that's pressure and competition and very ambitious people who have shitty morals" [3]. Gordon also noted that antibody treatments targeting beta-amyloid haven't stopped disease progression [3].
Gordon did not name the study behind the 70 percent figure in this conversation. The same episode also contains claims that deserve the skepticism the episode recommends. Gordon says "95% of the cases of Alzheimer's disease appear to be due to trauma and aging" [3]. He says his protocol put a patient's multiple sclerosis into remission in 90 days, and that boxer Gerald McClellan is about 20 percent better after four months of hormone replacement and a peptide [3]. Rogan cites ibogaine figures he has heard of: about 80 percent success with one session and 93 to 97 percent with two [3]. The summary of the episode itself says some of Gordon's reports are his personal reports rather than established findings, and that many claims come from his clinical experience [3]. Gordon also explains why his central idea is hard to test: "There's no radiological or neuro radiological procedure that will allow you to look at inflammation in the brain" [3]. An idea that can't yet be measured directly may still be right, but single cases, percentages without a named study and hard-to-measure mechanisms call for the same questions Attia asks of the metformin data.
Huberman on placebo: why controls exist, and where expectation stops
Huberman's solo episode on placebo gives the underlying reason every health claim needs a comparison group [4]. He argues that expectation is not imaginary. It works through the prefrontal cortex, which sends signals to the hypothalamus, brainstem and body, shifting dopamine, hormones, pain and heart rate [4]. In Parkinson's disease, some patients given a sugar pill showed real increases in dopamine release on PET imaging with raclopride [4]. In a conditioning study, saline injections raised growth hormone and lowered cortisol after people had first received a drug that does that. It happened even when they were told the injection would decrease growth hormone [4]. "So are we all just like salivating dogs? I guess we are," he says [4].
The packaging of a treatment changes its effect too. A placebo with a brand name works better than a generic one. Capsules beat tablets, injections beat pills, and an inert medical machine that hums and beeps produces the largest placebo effect of all [4]. Pill color mattered in three separate experiments: blue for sleep, red for stimulation, yellow for antidepressant effects [4]. For anyone judging a health claim, this is the key point. If the look, the price, the ritual and the story of a treatment can produce real effects by themselves, then a before-and-after story or a testimonial can't tell you whether the treatment worked. That is exactly why, as Huberman says, "placebo controls are vital for certain clinical studies" [4]. Separating a drug's real effect from expectation is the reason trials use them [4].
He is just as clear about the limits. In cancer care, people told a placebo will ease treatment often have less pain and nausea, but "those placebo effects are not acting directly on tumors to reduce their size or eliminate them" [4]. In an asthma study from Ted Kaptchuk's lab, published in the New England Journal of Medicine, people on placebo felt less discomfort, but their breathing patterns didn't change. Those on the real drug improved on both [4]. His conclusion: "Our beliefs and expectations are powerful as evidenced by the placebo effect itself, but they are not what we call omnipotent. They can't do anything and everything" [4]. He explains the boundary with anatomy. As far as we know, there is no direct pathway from prefrontal thoughts to muscle growth, but there are pathways to the hypothalamus and brainstem that control blood pressure and stress responses [4]. Responses also vary between people. The first formal placebo study found about 30 percent of participants had a robust response, and variation in the COMT gene correlates with how much a person responds [4].
The GIST of every new summary in your inbox, daily or weekly. Free, and you can unsubscribe any time.
Belief effects that scale, and a study read two ways
Huberman presented the same nicotine study in two episodes, and comparing the two is a lesson in itself. Experienced smokers vaped nicotine they were told was a low, medium or high dose. In reality everyone got the same amount, which was verified [2][4]. In the placebo episode he calls it "One of my all-time favorite studies." He says those told medium "performed better on a cognitive task" than those told low, that those told high did best, and that activity in the relevant brain region "scaled up according to whether or not people were told they got a small, medium or high amount of nicotine" [4].
In the journal club with Attia, the same study got a harder look [2]. It was a bioRxiv preprint, not yet peer reviewed. Thalamus activation scaled with the believed dose but was statistically significant only between low and high. The correlation in the thalamus figure was weak, with an r of about 0.27. The sample was powered for about 20 people per group, so smaller differences may have been missed. Smokers coughing in the scanner added movement artifacts. And The biggest missing control is a group given zero nicotine but told they got a high dose [2]. Huberman still concluded it was likely the first study to show that belief effects can be dose-dependent: "your brain is a prediction making machine" [2]. Together, the two readings show a result can be interesting and preliminary at the same time.
Alia Crum's studies, which Huberman describes in both episodes, follow the same pattern [2][4]. In Mind Over Milkshakes, people drank the same 380-calorie shake twice. It was labeled once as a 620-calorie "indulgent shake and once as a 140-calorie sensible shake. Ghrelin dropped more steeply when people believed it was indulgent [4]. What we believe about the foods we are consuming strongly impacts the downstream hormonal effects of consuming those foods [4]. The study was prompted by her advisor's question, What if all the effects of exercise are placebo [2]. Hotel housekeepers told their work counted as exercise improved blood pressure, resting heart rate and body weight [4]. In the journal club, Huberman put the weight difference at 12% more, without moving more" [2]. Attia raised the flip side, the nocebo effect: some people read a drug's list of side effects and seem to develop every one [2].
Where They Agree, and Where They Don't
Agreement: randomization is the gold standard, and associations are where you start. Attia says reverse causality "can only be answered through either direct randomization or Mendelian randomization" [1]. On metformin, "absent a randomized control trial. We will never know the answer" [2]. Huberman's placebo episode reaches the same place from another direction: expectation alone produces real effects, so only a controlled comparison can isolate a treatment's contribution [4]. Rogan and Gordon's discussion adds that even published results need to be reproduced before they can be trusted [3].
Agreement: skepticism is a tool for weighing evidence. Both journal-club hosts end up recommending light habits despite the correlational data [1]. Attia still prescribes metformin to insulin-resistant patients [2]. Huberman calls belief effects real while insisting they are "not what we call omnipotent" [4]. The shared stance is calibrated trust: act where the evidence is consistent, plausible and cheap to act on, and hold back where it isn't.
Disagreement: how much of the light-mood link is causal. Huberman estimates 65 to 75 percent and flags his own bias [1]. Attia, the more habitual skeptic of epidemiology, goes higher, as far as 80/20, because of the consistent dose-response and strong biological plausibility under the Bradford Hill criteria [1]. The skeptic is more confident here because the evidence meets specific criteria, not because of enthusiasm.
Disagreement: how firm the nicotine result is. In his solo episode, Huberman presents the belief study as a clean demonstration of dose-dependent effects on performance and brain activity [4]. In the journal club, he and Attia add the weak correlation, the small sample, the preprint status and the missing zero-nicotine control [2]. Neither account is wrong. They show how much a finding's apparent strength depends on who is presenting it and how closely they look.
Tension: the reproducibility number and the people who cite it. Gordon cites a 70 percent failure rate to reproduce published work, without naming the source in this conversation [3]. In the same episode he makes strong clinical claims based on individual patients [3]. Attia's approach suggests holding every claim to the same standard: what was compared, how many people, and what's missing. That includes claims made by people warning about bad science.
Tension: association studies and the headlines built on them. Huberman thinks the light paper's title misleads [1]. Attia thinks huge samples can make trivial differences look significant [1]. Gordon and Rogan tie bad science to incentives [3]. Huberman shows the actual conclusion can be the opposite of how a title reads [1]. Together they explain why headlines so often overreach.
Why It Works
The sources give specific reasons why these habits protect readers.
Reverse causality and confounding create associations without cause. A condition can drive the behavior that seems to cause it. Obesity can lead to diet soda, and depression can keep people indoors and awake at night [1]. People who make one healthy choice often make many others that are hard to measure [2]. Treatment groups can differ in ways matching can't fix, such as medication use [2]. Randomization breaks these links because chance, not the person's condition or choices, decides who gets what [1][2].
Analytic choices can manufacture results. Informative censoring removed exactly the people most likely to do badly from the metformin group. That produced a 15 percent apparent survival benefit, which disappeared in a larger, twin-matched analysis [2]. Such choices are often buried in methods sections or supplements [2]. That is why Attia reads beyond the figures when a field is unfamiliar [2].
Sample size cuts both ways. A small study can miss real differences it wasn't powered to detect [2]. A huge one can make differences that don't matter clinically look statistically significant [1]. So a null result only rules out effects as large as the study was powered to find [2], and a significant result still needs its effect size checked.
Expectation produces real, measurable effects. Placebos raise dopamine in Parkinson's patients [4], shift growth hormone and cortisol through conditioning [4], change ghrelin after a milkshake [4] and scale brain activity with a believed dose [2][4]. These run through circuits from the prefrontal cortex to the hypothalamus and brainstem [4]. Without a placebo-controlled comparison, a treatment's apparent benefit can't be separated from the benefit of believing in it [4].
Expectation has hard limits. The prefrontal cortex has a finite set of outputs, and none reach tumors directly [4]. Asthma patients on placebo felt better while breathing no better [4]. So there's a pattern to watch for: when a claimed benefit is subjective (pain, energy, mood, discomfort), expectation can explain a lot of it. When it is structural (tumor size, airflow, muscle growth, survival), expectation is much less likely to account for it [4].
Incentives and fraud corrupt the record. Pressure to publish positive results for funding encourages cutting corners [3]. In the Masliah case, reused and apparently falsified images across 132 papers put "a remarkably large body of scientific work" in doubt [3]. That happened in a field backed by billions in public funding [3]. Independent replication is the check. Without it, a published result is one claim, not settled fact [3].
Consistency and plausibility build trust. Attia's willingness to believe the light data rests on a dose-response pattern that holds across three statistical models, results that barely changed across them, and a known biological pathway from melanopsin cells to mood centers [1]. The ITP earns trust by running each test in three labs with genetically diverse mice [2]. Evidence that holds up across methods, labs and models deserves more weight than a single striking number.
Where It Applies, and Where It Doesn't
It applies most to observational health claims. Headlines like X is linked to lower depression risk or people who take Y live longer come from studies where reverse causality and confounding can't be ruled out [1][2]. They are useful starting points, but they don't prove cause.
It applies to single, small or unreplicated studies. A preprint with about 20 people per group and a weak correlation [2], a retracted circadian study [1] and a single patient's recovery [3] are all reasons for interest, not conclusions.
It applies to any treatment whose benefit is subjective. Supplements, devices, protocols and drugs judged by how people feel are exactly where brand, color, price and ritual can add real effects that don't come from the treatment itself [4].
It applies less when the cost of acting is low and the evidence lines up. Attia and Huberman both recommend getting daylight and keeping nights dim despite the correlational data. The dose-response is consistent, the biology is plausible and the habits are a low lift [1]. Skepticism is for sizing up a claim. It doesn't mean waiting for perfect evidence before doing something cheap and safe.
It applies less where good trials already exist. The ipilimumab trial was randomized, reported overall survival and showed a long-term tail of survivors [1]. Questions still remain, such as the weaker benefit in women and the autoimmune side effects [1]. But the basic cause-and-effect question has been answered in a way no association study can match.
Belief effects don't replace treatment. Placebo can ease the discomfort of chemotherapy or asthma but can't shrink tumors or restore breathing [4]. Behavioral practices like stress reduction, sleep and social support are, Huberman says, not placebo effects but real effects that can matter alongside actual treatment [4]. For anything serious, work with a qualified clinician rather than relying on belief, a headline or a single study.
Skepticism has to apply to everyone. The warning that most findings can't be reproduced [3] applies equally to clinical anecdotes, percentages without a named source and theories that can't yet be measured directly [3]. Reading evidence well means using the same standard for claims you like and claims you don't.
What To Do With It
Read past the headline and the title. Find the paper's actual conclusion. Huberman's light paper had a title that sounded like the opposite of its finding [1]. Figure headers tell you "what they want you to think they found," so check them against the data [2].
Ask Huberman's four questions. What question did they ask? What method did they use? What did they find? Do the conclusions actually follow from the findings? The last question is the one that matters most [2].
Test every association for reverse causality and confounding. Ask whether the condition could be causing the behavior, as with obesity and diet soda or depression and light [1]. Ask whether the people who chose the behavior differ in ways nobody measured [2]. If a study is only observational, treat it as a lead until randomized or Mendelian randomization evidence exists [1].
Check the baseline table and who was left out. Look at Table 1 to see whether the groups really match, for example on medications [2]. Ask whether people who quit, dropped out or got sicker were removed from the analysis, the informative censoring that made metformin look life-extending [2]. Look in the supplements, where the key analyses often sit [2].
Translate the statistics into plain terms. Subtract one from an odds or hazard ratio to get the percent change. Check whether the 95 percent confidence interval crosses one [1][2]. Ask how big the effect is and what the study was powered to detect. A small study can miss real effects, and a huge one can make trivial ones look significant [1][2]. For survival, look at the tail of the curve as well as the median [1].
Look for the missing control. Ask what the treatment was compared against. For anything judged by how people feel, ask whether there was a placebo arm, and remember that brand names, capsules, injections and humming machines all boost placebo response [4]. In the nicotine study, the missing zero-dose group was the biggest gap [2].
Ask whether expectation could plausibly produce the outcome. Subjective relief, pain, energy and some hormonal responses can move with belief. Tumor size and airflow in asthma did not in the studies Huberman describes [4]. The more a claim depends on how people feel, the more it needs a placebo comparison.
Wait for replication, and weigh consistency. One striking study, a preprint or a single patient's story isn't settled evidence [2][3]. Give more weight to findings that hold across models, labs and designs, like the dose-response that held across three statistical models in the light study [1] or results replicated across three labs in the ITP [2]. Remember that even papers from prestigious labs have turned out to contain apparently falsified data [3].
Hold people and sources to the same standard, including yourself. Note conflicts and biases, as Huberman did about his own research and the light device [1]. Discount unnamed statistics and personal success stories, whoever offers them [3].
Match your response to the stakes. For cheap, low-risk habits backed by consistent, plausible evidence, such as morning daylight and dim nights, acting now is reasonable [1]. For drugs, supplements and anything with side effects or cost, wait for stronger evidence and talk to a physician before changing what you take [2].
Sources
- Journal Club with Dr. Peter Attia: Effects of Light & Dark on Mental Health & Treatments for Cancer · Huberman Lab
- Journal Club with Dr. Peter Attia: Metformin for Longevity & The Power of Belief Effects · Huberman Lab
- Dr. Mark Gordon · The Joe Rogan Experience
- How Placebo Effects Work to Change Our Biology & Psychology · Huberman Lab