WHY META STUDIES ARE BULLSHIT
“meta-study = one person's opinion on which studies count.”
The Lesson
Meta-studies claim to combine flawed studies into reliable conclusions. The theory: small problems in individual studies average out. The reality: someone decides which studies to include. If they exclude one large study as 'garbage,' that single decision determines the entire outcome. The 'study of studies' becomes a study of one person's opinion about which studies are valid. That's not science-it's opinion disguised as science. The only valid meta-study: multiple studies with identical designs but insufficient sample sizes, combined for statistical power. Different designs with subjective inclusion? Don't believe it.
Real-World Example
A founder evaluating a health claim finds a meta-study supporting it. They look deeper: who decided which studies to include? One researcher excluded the largest study because of 'methodological concerns.' That study showed opposite results. Without it, the meta-study 'proves' the claim. With it, the claim fails. The founder realizes: the meta-study was one person's opinion with extra steps.
Watch Scott explain this lesson
Watch Lesson on Locals →