2023 Innovator of the Year: As AI models are released into the wild, Sharon Li wants to ensure they’re safe

Sharon Yixuan Li - SARA STATHAS

2023 Innovator of the Year: As AI models are released into the wild, Sharon Li wants to ensure they’re safe
MIT Technology Review, September 12, 2023
Artificial Intelligence
by Melissa Heikkilä

“Li’s research could prevent AI models from failing catastrophically when they encounter unfamiliar scenarios.”


As we launch AI systems from the lab into the real world, we need to be prepared for these systems to break in surprising and catastrophic ways. It’s already happening. Last year, for example, a chess-playing robot arm in Moscow fractured the finger of a seven-year-old boy. The robot grabbed the boy’s finger as he was moving a chess piece and let go only after nearby adults managed to pry open its claws.


This did not happen because the robot was programmed to do harm. It was because the robot was overly confident that the boy’s finger was a chess piece.


The incident is a classic example of something Sharon Li, 32, wants to prevent. Li, an assistant professor at the University of Wisconsin, Madison, is a pioneer in an AI safety feature called out-of-distribution (OOD) detection. This feature, she says, helps AI models determine when they should abstain from action if faced with something they weren’t trained on.


Li developed one of the first algorithms on out-of-distribution detection for deep neural networks. Google has since set up a dedicated team to integrate OOD detection into its products. Last year, Li’s theoretical analysis of OOD detection was chosen from over 10,000 submissions as an outstanding paper by NeurIPS, one of the most prestigious AI conferences.


We’re currently in an AI gold rush, and tech companies are racing to release their AI models. But most of today’s models are trained to identify specific things and often fail when they encounter the unfamiliar scenarios typical of the messy, unpredictable real world. Their inability to reliably understand what they “know” and what they don’t “know” is the weakness behind many AI disasters.

Read the Full Article »

About the Author:

Sharon Li is one of MIT Technology Review’s 2023 Innovators Under 35.