BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//UM//UM*Events//EN
CALSCALE:GREGORIAN
BEGIN:VTIMEZONE
TZID:America/Detroit
TZURL:http://tzurl.org/zoneinfo/America/Detroit
X-LIC-LOCATION:America/Detroit
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20070311T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20071104T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260930T102735
DTSTART;TZID=America/Detroit:20261016T100000
DTEND;TZID=America/Detroit:20261016T110000
SUMMARY:Workshop / Seminar:Anytime-Valid Confidence Sequences: Stopping Experiments Early with Netflix
DESCRIPTION:Randomized experiments have become the standard method for companies to evaluate the performance of new products or services. Beyond aiding managerial decision-making\, experiments mitigate risk by limiting the proportion of customers exposed to innovations. Since many experiments are conducted sequentially over time\, an emerging strategy to further de-risk the process is to allow managers to ``peek'' at the results as new data become available and stop the test if the results are statistically significant. The class of statistical methods that allow managers to peek and still provide valid inference are often called anytime-valid since they maintain proper uniform type-1 error guarantees. In this paper\, we extend existing anytime-valid approaches to accommodate the more complex yet standard settings in time series\, switchback\, and panel experiments. To achieve this\, we leverage the design-based approach to focus on assumption-light and managerial relevant finite-sample estimates defined on the study participants as a direct measure of the risks incurred by companies. As a special case\, our (asymptotic) results also provide a robust method for achieving always-valid inference in A/B tests. We further provide a variance reduction technique incorporating modeling assumptions and covariates. Finally\, we demonstrate the effectiveness of our proposed approach through a simulation study and three real-world applications from Netflix. Our results show that using our confidence sequence\, harmful experiments could be stopped after only observing a handful of units\; for instance\, our method would have stopped a 30\,000 person Netflix experiment after the first 100 people.
UID:153096-21915074@events.umich.edu
URL:https://events.umich.edu/event/153096
CLASS:PUBLIC
STATUS:CONFIRMED
CATEGORIES:Aem Featured,Graduate And Professional Students,Lecture,seminar,symposium,Talk
LOCATION:West Hall - 340
CONTACT:
END:VEVENT
END:VCALENDAR