ENBIS/JMP webinar: Test Selection as a Design of Experiments Problem
Test Selection as a Design of Experiments Problem
Speaker:
Ryan Lekivetz (JMP, Director of Advanced Analytics R&D)
Date: Nov 5th, 3 pm CET
Abstract:
Testing complex systems presents unique challenges. The space of possible inputs, options, configurations, and workflows can be enormous, making exhaustive testing impractical. At the same time, developers and users often have limited time and resources for formal testing activities.
This talk explores how principles from Design of Experiments (DOE) can be applied to the test case selection problem. By viewing test cases as experimental runs and system inputs as factors, DOE provides a structured framework for constructing efficient test suites that maximize information while minimizing effort. The discussion highlights connections between experimental design and software testing, demonstrating how statistical thinking can improve test coverage, identify defects more effectively, and increase confidence in system reliability.
Examples are drawn primarily from statistical software testing, but the same ideas apply more broadly to other complex systems, including AI tools and large language models, where interactions among inputs can create an overwhelming number of possible test scenarios.
Bio:
Ryan Lekivetz is a Director of Advanced Analytics R&D at JMP, heading the Design of Experiments (DOE) and Reliability Development team. Lekivetz earned his doctorate in statistics from Simon Fraser University in Burnaby, British Columbia. He has published papers on DOE topics in peer-reviewed journals and holds many patents that he shares with his team members.