Deborah Raji (EngSci 1T8 + PEY)

Beyond the Benchmarking Paradigm: Evaluation and Accountability for Machine Learning in Deployment

What does it mean for machine learning to work? As machine learning products get integrated into more and more consequential settings, it has become increasingly clear just how difficult it is to anticipate and measure how these models will behave at scale, in the real world.

In this talk, I'll discuss the existing norms around machine learning development and deployment, and lessons learnt in how to evolve past those current practices in order to better address the legal challenges and operational failure modes prevalent in real world systems, especially for the most marginalized impacted populations.

Speaker bio:

Deborah is a researcher at UC Berkeley interested in algorithmic auditing. She has worked closely with industry, government, civil society and within academia to push forward norms in ML practice on evaluation, documentation and accountability.

In 2020, just one year after graduating from EngSci, Deborah was named to MIT Technology Review’s Top Innovators Under 35.  She was named one Forbes Magazine's 30 Under 30 in 2021 and one of Time Magazine's 100 Most Influential People in AI in 2023.

Her research, which she began as an undergraduate EngSci student, was featured in a NY Times story in 2019.

Read Deborah's article "AI's Present Matters More Than ITS Imagined Future" in The Atlantic Magazine.