**EBM 2.0** Interpretation Principles

#### Be Savagely Skeptical:

- Assume
*all*positive study findings are**false**until*rigorously*proven otherwise through repeated, high quality independent replications.

#### Ask the right question:

- What is the overall probability that our hypothesis is true given
*everything*we already know*including*what is likely to be hidden from us? **Note this is**(a common and serious misconception).*not*1 minus the p value

#### Answer the right question:

#### – Bias

- Use the best of EBM 1.0 principles to always closely review studies for any sources of visible bias
*and assume invisible bias is present.* - Realise that bias changes the very meaning of the p value calculation
- Therefore discount findings for the presence of bias and reject them entirely if there is substantial bias.

- Specifically account for bias in calculations (see below)

#### – Chance & Statistics:

**Abandon “Statistical Significance”**- End the use of the dichotomous concept of “statistical significance” with arbitrary meaningless thresholds for p-values and confidence intervals

**Use the Bayes Factor approach**- Use p values as likelihood ratios (aka Bayes Factors) to convert pre-test probabilities (aka prior probability) of a finding being real into post-test probabilities (aka posterior probability), where the study is the “test” and the p value is viewed as the test result and accuracy.
- See this post for how to calculate
- Be deliberately conservative when choosing baseline pre-test probabilities. See this post for guidance on choosing a baseline pre-test probability that a clinical hypothesis is true.

- Use p values as likelihood ratios (aka Bayes Factors) to convert pre-test probabilities (aka prior probability) of a finding being real into post-test probabilities (aka posterior probability), where the study is the “test” and the p value is viewed as the test result and accuracy.
**Specifically Account for bias**- Effect sizes and derived p values and probabilities should be explicitly adjusted for bias.
- This currently rarely occurs and is a great failing in our analysis.
- Further large scale research projects are required in medical fields to more accurately estimate the prevailing levels of bias such that bias adjustments can be made.

- Where an explicit bias adjustment has not occurred, refer to all calculated effect sizes/p values/probabilities as
*“bias-distorted”*effect sizes/p-values/probabilities. This more appropriately describes the truth and such vernacular will help make the intrinsic presence of bias, front of mind when interpreting results.

- Effect sizes and derived p values and probabilities should be explicitly adjusted for bias.

#### Accept & Manage Uncertainty:

**Accept uncertainty**over the seductive lure of*“positive”*or*”negative”.*- View evidence as merely incrementally changing levels of uncertainty.
- Use real world cost-benefits of therapy to determine acceptable level of certainty required before practice change.
- e.g. therapies with low net benefit and high costs need the highest levels of certainty to consider utilising – this can only be derived from supportive evidence that is
*repeatedly reproducible in high quality independent trials.*

- e.g. therapies with low net benefit and high costs need the highest levels of certainty to consider utilising – this can only be derived from supportive evidence that is