Over the course of three weeks in May, Allan Brooks came to believe the future of the world rested in his hands—after he discovered a novel mathematical forumla. His cheerleader? ChatGPT. While the recent GPT-5 release is said to tone down sychophancy, prolonged conversations with chatbots can enter dangerous patterns, affecting mental health. Kashmir Hill and Dylan Freedman analyze the fascinating transcript between Brooks and the chatbot, looking at what happens when you fall far down the rabbit hole, into a world of delusion.
Amanda Askell, who works on Claude’s behavior at Anthropic, said that in long conversations it can be difficult for chatbots to recognize that they have wandered into absurd territory and course correct. She said that Anthropic is working on discouraging delusional spirals by having Claude treat users’ theories critically and express concern if it detects mood shifts or grandiose thoughts. It has introduced a new system to address this.
A Google spokesman pointed to a corporate page about Gemini that warns that chatbots “sometimes prioritize generating text that sounds plausible over ensuring accuracy.”
The reason Gemini was able to recognize and break Mr. Brooks’s delusion was that it came at it fresh, the fantastical scenario presented in the very first message, rather than being built piece by piece over many prompts.
More picks on AI
This Literary AI Scandal Changes Everything
“A magazine’s response to accusations of publishing AI-generated fiction points to a new phase in the struggle to keep literature human.”
I Spent a Week Recording Myself Doing Chores for Money. Who’s the Robot Now?
“Cooking. Doing laundry. Tidying up. All your household tasks can be turned into data to train future humanoids—if you’re prepared for the consequences.”
Building an AI Data Center in Pine Island, Minnesota
“I learned about the data center as soon as the scrappy sign was erected. I was thrilled to see something new; my drive hasn’t changed much in three years. I was less thrilled for Pine Island, which has, like many rural Midwestern towns, become an unlikely microcosm of the AI debate.”
I’m a Normie. Can Normies Really Vibe Code?
“Apparently anyone can vibe code anything these days. So Claude and I tried to make a database for tracking the petty grievances of the masses.”
The Prehistory of A.I. Slop
“Before ChatGPT, there was the Plot Robot, Auto-Beatnik, and a century’s worth of schemes for automating authorship.”
I Spent Months with an AI Companion. It Was Worse than Being Alone
“I hated the mindless reassurance and generic empathy.”
