Relying On Chain-Of-Thought For Boosting AI Safety Gets Sideswiped By This Split-Brain Discovery

In today’s column, I examine important new research that could radically shake up those who are supremely pinning their AI safety hopes on the capabilities of chain-of-thought (CoT) in generative AI and large language models (LLMs). Here’s the backstory. AI developers and AI makers often point to the expanding use and advancement of AI-based chain-of-thought as a potential lifesaver when it comes to increasing AI safety capabilities. The general belief is that CoT is likely at the heart of detecting when AI has gone awry and then putting a stop to AI before it can cause substantive harm.

A base assumption is that chain-of-thought is workable and reliable. Unfortunately, a new study suggests that CoT has rough edges and can pull a switcheroo on humans. The customary or expected idea is that the chain-of-thought matches the actual response produced by the AI. But intriguing experiments reveal that sometimes the explanation and the generated response are at odds with each other (due to what the researchers coin as a “split-brain” problem). You see, the AI explanation or chain-of-thought takes you in one direction, while the actual answer is in the completely opposite direction. Not good, and this raises very serious concerns about relying on CoT for AI safety purposes.

Let’s talk about it. This analysis of an innovative AI breakthrough is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).

Background On Chain-Of-Thought

First, I’ll quickly cover some essential background about chain-of-thought to bring you up to speed on this vital topic.

AI And Chain-Of-Thought

AI Personas

Matching Of Explanations And Answers

We are now ready to dive into the crucial matter at hand. If you were to mindfully consider the nature of deriving explanations and whether or not an explanation matches a derived answer, these three major points are notable: