Speaker
Description
In comparative statistical tests of parallel treatment groups, a new drug is commonly considered superior to the current version if the results are statistically significant. Significance is then based on confidence intervals and p-values, the reporting of which is requested by most top-level medical journals. However, in recent years there have been ongoing debates on the usefulness of these parameters, leading to a ‘significance crisis’ in science.
We will show that this conventional quest for statistical significance can lead to confusing and misleading conclusions for the patient, as it focuses on the average difference between treatment groups. By contrast, prediction or tolerance intervals deliver information on the individual patient level, and allow a clear interpretation following both frequentist and Bayesian paradigms.
Additionally, treatment successes on the patient level can be compared using the concept of individual superiority probability (ISP). While a p-value for mean treatment effects converges to 0 or 1 when the sample size gets large, the ISP is shown to be independent of the sample size, which constitutes a major advantage over the conventional concept of statistical significance. The relationship between p-values, ISP, confidence intervals and tolerance intervals will be discussed and illustrated with analysis of some real world data sets.
| Keywords | significance, p-value, pharma |
|---|