Over the course of three weeks in May, Allan Brooks came to believe the future of the world rested in his hands—after he discovered a novel mathematical forumla. His cheerleader? ChatGPT. While the recent GPT-5 release is said to tone down sychophancy, prolonged conversations with chatbots can enter dangerous patterns, affecting mental health. Kashmir Hill and Dylan Freedman analyze the fascinating transcript between Brooks and the chatbot, looking at what happens when you fall far down the rabbit hole, into a world of delusion.
Amanda Askell, who works on Claude’s behavior at Anthropic, said that in long conversations it can be difficult for chatbots to recognize that they have wandered into absurd territory and course correct. She said that Anthropic is working on discouraging delusional spirals by having Claude treat users’ theories critically and express concern if it detects mood shifts or grandiose thoughts. It has introduced a new system to address this.
A Google spokesman pointed to a corporate page about Gemini that warns that chatbots “sometimes prioritize generating text that sounds plausible over ensuring accuracy.”
The reason Gemini was able to recognize and break Mr. Brooks’s delusion was that it came at it fresh, the fantastical scenario presented in the very first message, rather than being built piece by piece over many prompts.
More picks on AI
Building an AI Data Center in Pine Island, Minnesota
“I learned about the data center as soon as the scrappy sign was erected. I was thrilled to see something new; my drive hasn’t changed much in three years. I was less thrilled for Pine Island, which has, like many rural Midwestern towns, become an unlikely microcosm of the AI debate.”
I’m a Normie. Can Normies Really Vibe Code?
“Apparently anyone can vibe code anything these days. So Claude and I tried to make a database for tracking the petty grievances of the masses.”
The Prehistory of A.I. Slop
“Before ChatGPT, there was the Plot Robot, Auto-Beatnik, and a century’s worth of schemes for automating authorship.”
I Spent Months with an AI Companion. It Was Worse than Being Alone
“I hated the mindless reassurance and generic empathy.”
Meet the Sad Wives of AI
“Are you married to a man who’s obsessed with AI? I’m so, so sorry.”
The Chinese Whiz Kids of Silicon Valley
“Chinese-born tech workers have fueled Silicon Valley for decades. In the AI era, they’re superstars.”
