Main content start

Statistical foundations of trustworthy AI engineering

Date
Tue January 28th 2025, 4:30pm
Location
Lane (01-200) 034
Speaker
Anastasios Angelopoulos, UC Berkeley

My research has been almost entirely devoted to a single question: How can we build trustworthy systems from untrustworthy AI algorithms? Answering this question is difficult because modern AI models can be wrong in unpredictable ways. From data, these models learn biases, spurious associations, and imperfect world-models that are difficult to debug due to their statistical nature. But to use AI in critical applications — from legal and financial institutions to power plants to hospitals, where safety, and lives are at stake — we need trust. Part of what holds us back is a lack of formally grounded but practical statistical methodology for ensuring that we are able to use AI reliably, even when the underlying model may have flaws.

The talk will have two halves. In the first half, I will discuss conformal risk control, a statistical framework for reliable decision-making using black-box models. In the second half, I will discuss AI evaluations for aligning AI with human preferences and safety. I will focus both on the foundational statistical methodology underlying these techniques and also the large-scale deployments that have resulted, and the opportunities for future research that arise.

Zoom Recording [SUNet/SSO authentication required]