ENBIS Webinar: Statistical Significance and p-values

Thursday 3 Feb 2022, 14:00 → 15:30 Europe/Amsterdam

Jean-Michel Poggi (University of Paris-Saclay)

ENBIS will dedicate this webinar to the memory of Sir David Cox, who sadly passed away in January 2022.

Daniel Lakens is an experimental psychologist working at the Human-Technology Interaction group at Eindhoven University of Technology. In addition to his empirical work in cognitive and social psychology, he works actively on improving research methods and statistical inferences, and has published on the importance of replication research, sequential analyses and equivalence testing, and frequentist statistics. He was involved in establishing dedicated grants for replication studies by the Dutch science funder NWO, and with Brian Nosek co-edited a special issue with the first Registered Reports in psychology in 2014. His lab is funded until 2022 by a VIDI grant on a project that aims to improve the reliability and efficiency of psychological science. He teaches about better research practices on Coursera, and received the Leamer-Rosenthal Prize for Open Social Science in 2017 for his course ‘Improving Your Statistical Inferences’ in which more than 50.000 learners have enrolled.

 

Bernard G Francq is Lead Statistician with GSK Biologicals, driving statistical innovation for CMC projects worldwide. He holds a PhD in Statistics (UCLouvain, 2013). His work on errors-in-variables (EIV) regressions in method comparison studies has been awarded Best MSc Thesis Biostatistics (Quételet 2008, Belgium), Best Chemometrician Prize (Chimiométrie 2009, Paris), and Best Young Researcher (Agrostat 2012, Paris). His communication skills have been recognized with the Greenfield Challenge Award (ENBIS 2012, Ljubljana). Recent work on tolerance intervals in bridging studies was awarded Best GSK Statistical Paper (2020). He lectures at UCLouvain and regularly offers trainings to statisticians in the (bio)pharmaceutical industry.

 

Stephen Senn has worked as a statistician but also as an academic in various positions in Switzerland, Scotland, England and Luxembourg. From 2011-2018 he was head of the Competence Center for Methodology and Statistics at the Luxembourg Institute of Health. He is the author of Cross-over Trials in Clinical Research (1993, 2002), Statistical Issues in Drug Development (1997, 2007,2021), Dicing with Death (2003). In 2009 was awarded the Bradford Hill Medal of the Royal Statistical Society. In 2017 he gave the Fisher Memorial Lecture. He is an honorary life member of PSI and ISCB.

 

Ron Kenett is Chairman of the KPA Group, Israel, Chairman of the Data Science Society at AEAI, Senior Research Fellow at the Samuel Neaman Institute, Technion, Haifa, Israel. and Research Professor at the University of Turin, Italy. Authored and co-authored over 250 papers and 16 books on applied statistics topics, Awarded the 2013 Greenfield Medal by the Royal Statistical Society and, in 2018, the Box Medal by the European Network for Business and Industrial Statistics. Ron is emphasizing the role of information quality as an objective of statistical analysis.

14:00 → 15:30 Statistical Significance and p-values

This meeting is organized to present and discuss the issues listed in the title. It consists of 3 shorts presentations and discussions of recognized experts. The objective is to both, provide an introduction and a review of a topic with current significant impact of the role of statistics in healthcare and beyond.

Trends towards significance
Stephen Senn

There are many valid criticisms of P-values but the criticism that they are largely responsible for the reproducibility crisis has been accepted rather lightly in some quarters. Whatever the inferential statistic that is used, it is quite illogical to assume that as the sample size increases it will tend to show more evidence against the null hypothesis. This applies to Bayesian posterior probabilities as much as it does to P-values. In the context of P-values it can be referred to as the trend towards significance fallacy but more generally, for reasons I shall explain, it could be referred to as the anticipated evidence fallacy.
The anticipated evidence fallacy is itself an example of the overstated evidence fallacy. I shall also discuss this fallacy and other relevant matters affecting reproducible science including the problem of false negatives.

p-value, s-value, B-value, D-value, … what else?
Tolerance intervals: Beyond the t-test and p-values
Bernard G Francq, Ron Kenett

The statistical significance is often based on confidence intervals (or credible intervals in Bayesian analysis) and p-values, the reporting of which is requested by most top-level medical journals. However, in recent years there have been ongoing debates on their usefulness, leading to a ‘significance crisis’ in science.
In this talk, some alternative solutions proposed in the literature like the s-value, D-value or B-value (namely, the probability that a patient under treatment A ends up with a better clinical outcome compared to another patient under treatment B) will be reviewed. We'll show that, in medical research, the treatment successes on the patient level can be elaborated using the concept of individual success probability (ISP) which generalizes the B-value. The ISP allows a more pragmatic interpretation under both, frequentist and Bayesian, paradigms.
The relationships between p-value, ISP and tolerance intervals will be discussed and illustrated with toy examples (1 sample t-test, paired t-test, cross-over trials, 2 samples t-test) and real-world data sets.

Speakers: Bernard Francq (GSK), Prof. Daniël Lakens (Eindhoven University of Technology, Netherlands), Prof. Ron Kenett (KPA Group and Samuel Neaman Institute, Technion, Israel), Stephen Senn (Statistical Consultant, Edinburgh, Scotland, United Kingdom)