Hallucinated Citations Are Polluting the Scientific Literature. What Can Be Done?

When tools like Chat-GPT first gained widespread popularity, their tendency to hallucinate citations and references led to shame for scientists and attorneys who slipped those citations into their own briefs and studies. The pattern was troubling, but also avoidable—a little extra due diligence could weed out the confabulated referents. But as Nature finds in a collaboration with the screening company Grounded AI, scientific studies are increasingly filled with hallucinations that are slipperier than they appear. A title that matches up with a cited author’s area of study; co-authors that have published together previously; even volume/page numbers that sync to the supposed journal and date of publication. The result is a piece of microfiction, an imagined work of research that is fully plausible, but doesn’t actually exist. Remember when we bemoaned the erosion of “truth” just one short decade ago? As it turns out, we ain’t seen nothin’ yet.

In some cases, including in references in published articles, all of the component parts are made up, says Kathryn Weber-Boer, director of scientometrics at the London-based company Digital Science …. AI also hallucinates DOIs, both in references that are otherwise genuine as well as in fabricated ones, she adds.

AI-generated references commonly combine fragments of genuine publications, say researchers who have studied the issue. Joe Shockman, co-founder and chief executive of Grounded AI, calls such references ‘Frankenstein’ citations, likening their assembly to that of the fictional monster. “It looks real to a human being, but is not actually a reference to a real thing,” says Shockman, who is based in Ashland, Oregon.