**The Talk – **2 Minute Pitch

**“Evidence Based Fraud”** & *The End of Statistical Significance.*

**– Why you need to watch this talk –**

**“Evidence Based Fraud”** & *The End of Statistical Significance.*

**– Why you need to watch this talk –**

**“Evidence Based Fraud”** & *The End of Statistical Significance.*

This talk will forever change the way you interpret Evidence Based Medicine.

**Most Published Research Findings are False (00:00-04:46)**

- Introduction to the problem – that most published research findings are false

**We screwed up the statistics (04:46 – 13:13)**

- Discusses the key reasons why statistics such as the p value and statistical significance did not mean what we thought it mean and shows how we consequently grossly over-estimated the likelihood of our findings being real.
- Discusses the key statements from the American Statistical Association regarding the p value
- Demonstrates how p values are really likelihood ratios that convert a pre-test probability of a hypothesis being true (aka prior probability) into a post-test probability (aka posterior probability) where the study is the “test” and the p value represents the accuracy of the “test”.

This is further explained in this supplementary post

**We underestimated the impact of bias (13:12-19:53)**

- Discusses the role of bias in creating false research findings
- Explains how bias doesn’t just affect our study findings but affects the validity of the p value itself and the post-test probability calcuations
- Explains how we have wilfully ignored the major source of bias: Invisible Bias
- Discusses how we must explicitly discount our post-test probabilities for both Invisible and Visible Bias.

**EBM 2.0 – The Way Forward 19:53-24:31**

- Briefly summarises the key principles of EBM 2.0 to take us forward and help protect and serve our patients best.
- Further details are provided in the full version of the talk and the supplementary posts on this webpage under EBM 2.0: More Information

**“Evidence Based Fraud”** & *The End of Statistical Significance.*

This talk will forever change the way you interpret Evidence Based Medicine.

While watching the whole talk provided above is recommended, this Chapter guide below will allow viewers to accelerate to parts that they believe will be of most value to them. Note that any references in the talk to “you” or “we” were intended to be inclusive of me and almost every clinician and every person involved in EBM in the world. We’ve all been mislead and it is not our fault – the talk explains why and provides a way forward – EBM 2.0.

**Introduction (0:00 – 08:40)**

- Introduction to the problem – that most published research findings are false

**Bias, the underestimated factor (08:40-26:46)**

- Discusses the role of bias in creating false research findings
- Provides several examples of both detectable and the more insidious undetectable forms of bias and argues that some bias needs to be “priced in” to all EBM and consequently our statistical tests and conclusions need to be discounted to account for this.

**Chance, and how we screwed up the statistics so, so badly (26:46-45:20)**

- Discusses the key reasons why statistics such as the p value and statistical significance did not mean what we thought it mean and shows how we consequently grossly over-estimated the likelihood of our findings being real.
- Discusses the key statements from the American Statistical Association regarding the p value
- Demonstrates how p values are really likelihood ratios that convert a pre-test probability of a hypothesis being true (aka prior probability) into a post-test probability (aka posterior probability) where the study is the “test” and the p value represents the accuracy of the “test”.
- This is further explained in this supplementary post

**The ATOM Principle (45:20-1:07:22)**

- Discusses the ATOM principle recommended by the American Statistical Association to guide a path forward for EBM with several examples of applying this principle provided. ATOM stands for:
- Accept Uncertainty and be
- Thoughtful
- Open
- Modest

**EBM 2.0 (1:07:22-1:20:46)**

- Describes the EBM 2.0 project and movement aiming to change the way we practice EBM forever.
- Summarise the key principles of EBM 2.0.
- EBM 2.0 as an update to EBM 1.0 incorporates its useful parts (e.g. diligently examining studies for bias) and replaces the non-functioning parts (e.g. the use of “statistical significance”) with a new approach.

**Recently this talk was presented at:**

- ACEM Western Australian Scientific Meeting 2019
- ACEM Annual Scientific Meeting 2019 – heavily abridged version in the “Festival of Dangerous Ideas”
- EMS Conference Rusutsu Japan, 2020
- WA Rural Health Conference, 2020 – virtual conference