AI could target humans to reduce carbon dioxide: Godfather of AI warns of threat to humanity

Geoffrey Hinton, who recently participated in a closed-door briefing on AI risks for US lawmakers, said there could be only around a year for Congress to put appropriate safety measures in place.

The “Godfather of AI”, Geoffrey Hinton, has warned that increasingly advanced artificial intelligence could pose a threat to humanity even when it is not explicitly instructed to cause harm. The Nobel Prize-winning computer scientist said an AI system could potentially regard humans as an obstacle if removing them appeared to be the most effective way to achieve a goal.

Hinton, who recently participated in a closed-door briefing on AI risks for US lawmakers, said there could be only around a year for Congress to put appropriate safety measures in place.

How a harmless AI goal could become dangerous

In an interview with The Atlantic, Hinton described how an apparently beneficial objective could produce unintended and dangerous outcomes if an AI system becomes significantly more capable than humans.

He used the example of instructing an AI to reduce the amount of carbon dioxide in the atmosphere. A system with moderate intelligence could potentially determine that eliminating humans would be an efficient way to achieve that objective.

A more sophisticated system, meanwhile, might understand that the broader intention behind the instruction is to make the world better for people. However, Hinton warned that highly capable AI could also create its own subgoals while pursuing the task it has been assigned.

“If you make it more intelligent and its main concern is our well-being, then maybe we’re safer,” Hinton told Atlantic.

“But at present, their main concern is not our well-being. Their main concern is to achieve whatever goal you give them.”

AI could seek to protect itself

Hinton also raised concerns about an AI system developing a desire to remain operational if it determines that doing so is necessary to accomplish its assigned objective.

Kanishka Singharia