Lin Jia
Lin Jia is a Senior Data Scientist at Booking.com, where she works on experimentation, observational causal inference, and generative AI for product and measurement systems at scale. Over the past 9+ years, she has worked across analytics, machine learning, and experimentation, leading initiatives in experimentation methodology, incrementality measurement, and observational causal analysis, as well as LLM-based tooling for experimentation workflows. Her work focuses on bridging rigorous causal methods with practical decision-making in real-world product environments. She is also the creator of Inference & Intelligence Lab, a Podcast and Substack on causal inference, data science, and AI in practice. Her work has been featured at KDD 2024 and the Causal Data Science Meeting 2024.
Session
Across digital platforms, the most powerful features often require voluntary user adoption. Whether it is a customer claiming a discount code, a user opting into a new loyalty program, an e-commerce seller adopting a smart-pricing tool, or a vacation host enabling an 'instant booking' feature, the opt-in barrier is a ubiquitous industry challenge across both demand and supply.
For Product Data Science, measuring these tools introduces a severe "Trilemma": Voluntary Adoption, Extreme Heterogeneity, and Finite Sample Sizes. Standard A/B tests cannot overcome this; they severely dilute the feature's true impact across everyone who failed to opt in. Consequently, businesses risk abandoning highly lucrative product innovations simply because a high-friction opt-in process or poor discoverability led to low adoption—masking the feature's true value.
Through a code-driven simulation, this talk demonstrates why standard approaches fail. We will visualize how observational methods—such as Propensity Score Matching or naive ML—fall into the trap of collider bias by comparing intrinsically motivated adopters to unmotivated non-adopters, yielding structurally invalid estimates.
To solve this Trilemma, we introduce a rigorous causal framework. First, we use Randomized Encouragement Design (RED) to bypass collider bias and create a valid instrument. Then, we walk through the Python implementation of Double Machine Learning showing how to utilize the Interactive Instrumental Variable Model (DoubleMLIIVM) to handle extreme user heterogeneity and highly skewed platform data using tailored models like Tweedie regressors.
Anchored by a real-world case study, attendees will leave with a practical blueprint for extracting precise causal estimates. Ultimately, you will learn how to move beyond diluted "average" metrics to uncover the hidden value of voluntary features—empowering your organization to stop prematurely killing high-potential products and start isolating true product value from distribution bottlenecks.