When tools like Chat-GPT first gained widespread popularity, their tendency to hallucinate citations and references led to shame for scientists and attorneys who slipped those citations into their own briefs and studies. The pattern was troubling, but also avoidable—a little extra due diligence could weed out the confabulated referents. But as Nature finds in a collaboration with the screening company Grounded AI, scientific studies are increasingly filled with hallucinations that are slipperier than they appear. A title that matches up with a cited author’s area of study; co-authors that have published together previously; even volume/page numbers that sync to the supposed journal and date of publication. The result is a piece of microfiction, an imagined work of research that is fully plausible, but doesn’t actually exist. Remember when we bemoaned the erosion of “truth” just one short decade ago? As it turns out, we ain’t seen nothin’ yet.
In some cases, including in references in published articles, all of the component parts are made up, says Kathryn Weber-Boer, director of scientometrics at the London-based company Digital Science …. AI also hallucinates DOIs, both in references that are otherwise genuine as well as in fabricated ones, she adds.
AI-generated references commonly combine fragments of genuine publications, say researchers who have studied the issue. Joe Shockman, co-founder and chief executive of Grounded AI, calls such references ‘Frankenstein’ citations, likening their assembly to that of the fictional monster. “It looks real to a human being, but is not actually a reference to a real thing,” says Shockman, who is based in Ashland, Oregon.
More picks about science
AI Is Dead. Organoids Are Alive
“Mini human brains are being grown in labs all over the world. Soon, they could outthink neural networks.”
The Soviets Drilled the Deepest Hole Ever
“What on Earth for?”
The Children a Scientist Took Home
Carleton Gajdusek won the Nobel Prize for his work on a disease in Papua New Guinea. But his biggest experiment was on the children he took from the field to the US—many of whom he sexually molested, with the knowledge and, in some cases, approval of his professional peers and colleagues: In his journals and…
What Would It Mean to See a New Color?
“Olo can be described as the ‘greenest green,’ though it can never be captured by a camera. But its development tells us a lot about our color vision.”
A Fatal Reaction
“A cutting-edge gene-editing trial in China went disastrously wrong. A family wants accountability.”
Gaza’s Deadly Soil
“Amid tight Israeli restrictions, new remote research is starting to make visible the scale of toxic fallout across the Gaza Strip.”
