2025-04-23 –, Helium3
This talk introduces a novel Python library for statistical testing using e-values, offering a refreshing alternative to traditional p-values. We'll explore how this approach enables real-time sequential testing, allowing data scientists to monitor experiments continuously without the statistical penalties of repeated testing. Through practical examples, we'll demonstrate how e-values provide more intuitive evidence measures and enable flexible stopping rules in A/B testing, clinical trials, and anomaly detection. The library implements cutting-edge methods from game-theoretic probability, making advanced sequential testing accessible to Python practitioners. Whether you're conducting A/B tests, monitoring production models, or running clinical trials, this talk will equip you with powerful new tools for sequential data analysis.
Modern data science demands flexible statistical methods that can handle sequential data analysis and continuous monitoring. Traditional p-values, while widely used, have limitations when dealing with sequential testing scenarios. This talk introduces a Python library that implements e-values and e-processes, offering a more natural approach to measuring statistical evidence and enabling true sequential testing.
Outline:
-
Introduction to E-Values
- The limitations of p-values in sequential testing
- What are e-values and why do they matter?
- Key advantages: intuitive interpretation, composability, and sequential validity
- Real-world applications: A/B testing, clinical trials, anomaly detection -
Library Overview and Core Features
- Architecture and design philosophy
- Key components:- Sequential testing framework
- Confidence sequences
- Standard (classical) testing with e-values
- Conformal e-testing
- Integration with existing Python data science tools
- Live coding demonstrations of basic usage
-
Practical Applications
- A/B testing with continuous monitoring
- Sequential hypothesis testing
- Adaptive experimental design
- Real-time anomaly detection
- Live demonstrations with real-world datasets -
Advanced Features and Extensions
- Conformal prediction integration
- CUSUM procedures for change detection
- Adaptive thresholding
- Power analysis for e-value tests
- Performance considerations and optimizations -
Best Practices and Guidelines
- When to use e-values vs. traditional methods
- Setting up monitoring frameworks
- Common pitfalls and how to avoid them
- Tips for production deployment -
Future Directions and Community Involvement
- Roadmap for future development
- How to contribute
- Resources for learning more
The talk will include interactive demonstrations and real-world examples, making complex statistical concepts accessible to practitioners. Attendees will leave with:
- Understanding of e-values and their advantages
- Practical knowledge of implementing sequential testing
- Hands-on experience with the library
- Best practices for production deployment
- Resources for further learning
Advanced
Expected audience expertise: Python:Novice
Public link to supporting material, e.g. videos, Github, etc.:
I am a Machine Learning Engineer at H&M Group, former Data Scientist at Lidl Sweden, as a professional I am designing Machine Learning services, extracting insights and arranging meaningful stories for my clients by conducting high-quality modeling, engineering, data mining and analytics.
I have a Bachelor degree in Statistics and Probability theory from Uppsala University of Sweden. Because I am a Statistician at core I have good experience with Data Sciencr, Python, R, time series modeling, simulations, machine learning algorithms, SQL, Excel, Spark and database technologies, as well as good communication skills.
You’ll find two comprehensive Python libraries I have open-sourced. One is based on an emerging modern statistical hypothesis testing framework using e-values and martingales based on game-theoretic statistics. The other is for computational Supply Chain and Logistics. The first one is called ’expectation’ and the second one is called ’supplyseer’ and you can find both on my GitHub.