Faculty Fast Take: Wait … the People Building AI Want to Slow It Down?

Anthropic lead researcher Evan Hubinger made headlines earlier this month after declaring there was a greater than 10% chance AI could wipe out humanity.

Should society be concerned?

The answer is potentially yes, according to Erik Noyes, associate professor at Babson College and director of The Generator at the C. Dean Metropoulos Institute for Technology and Entrepreneurship, within the Arthur M. Blank School for Entrepreneurial Leadership.

“It’s a person leading alignment science at a frontier lab saying his company doesn’t have a robust plan,” Noyes said. “He sees risk from today’s models as low but his worry is about future systems, which will improve themselves through recursive self-improvement.”

After Hubinger’s comments made the rounds, Anthropic, OpenAI, and SpaceXAI respective leaders Dario Amodei, Sam Altman, and Elon Musk collectively agreed to slow the development of the technology before it reaches a point where it can no longer be controlled.

How might this be accomplished? Through independent evaluators with employee-like access and shared safety standards, Amodei proposed, which also received backing from Altman and Musk.

“How often do you see industries ask for intrusive oversight?” said Noyes, who also is the Michael London ’92 and Stephen H. Kramer ’92 Term Chair in Entrepreneurship. “It’s a sign of the complex times we’re in.”

Yet in order to truly manage risk, Noyes says international oversight is required, as competing nations around the world seek to harness what’s possible.

“This should be a focus of international governmental coordination,” he says. “AI is a historic class of technology, and it needs to be managed for global good.”