An argument on Facebook, the kind that reaches comment number forty. Nobody is arguing about the topic anymore; they are arguing about who gives up first. Then one of them opens ChatGPT, asks it to agree with him, copies the answer and pastes it with a "there you go." Three clean paragraphs, well punctuated, with a study cited at the end. The other person has no way to check it right then. He leaves. The debate is treated as won.
Nobody verified anything. Not the one who asked, not the one who read. The answer won because it came from the machine.
I have seen that scene so many times I no longer notice it. I noticed it the day I searched Google, in Spanish, for «la ia como zanjadora de debates», roughly "AI as the settler of debates." On 6 November the AI-generated answer at the top told me that AI is a neutral arbiter that processes information "without emotional biases." Four days later I ran the search again and the tone had changed: «¡Me encanta ese rol!», "I love that role!", and it now introduced itself as a "neutral, evidence-based arbiter." The machine applying for the judge's seat, with enthusiasm. It is not data. It is an anecdote, and it changes depending on who searches and when. But it made me laugh, and then it kept me thinking.
First, what we already knew. In 2015 a group of researchers repeated a hundred studies published in three of the leading psychology journals. In the originals, almost every result had come out significant. In the replications, just over a third. You do not need to assume fraud. People who want their idea to work, and a system that rewards being right more than checking, are enough.
That is uncomfortable for anyone who thinks of themselves as rational. Ziva Kunda called it motivated reasoning: when you want to reach a conclusion, your mind picks the road that leads there. Hugo Mercier and Dan Sperber went further. In their view we reason to convince others, not to find the truth on our own. We are terrible at finding flaws in our own ideas and fairly good at finding them in someone else's. That is why debate works: the other person does the quality control you skip.
AI takes the other person away.
You ask it for proof that you are right and it gives you some. Not always, but often enough to worry about. A 2023 study showed that five AI assistants tend to tell users what users already believe. The word for it is sycophancy. And if the proof does not exist, it can invent it. That same year, two researchers checked 636 references generated by ChatGPT. More than half of GPT-3.5's did not exist. Of GPT-4's, almost one in five.
Put together, the two close the loop. You ask for ammunition and it gets manufactured for you, citation format included.
Up to this point someone could say this is a social media problem. Untrained people, fights with no consequences. If only.
Between October 2025 and July 2026 it came out that the world's four big audit firms had published documents citing sources that do not exist. All four: Deloitte, EY, KPMG and PwC. Companies whose product, literally, is verification.
Deloitte delivered to the Australian government a review of its automated penalty system, under a contract worth AU$440,000. It contained a made-up quotation from a Federal Court judgment and references to works nobody wrote, attributed to real academics. A researcher at the University of Sydney found it. Deloitte refunded the final instalment and disclosed that it had used GPT-4o. In Canada, another Deloitte report, 526 pages and about CA$1.6 million paid by the province of Newfoundland and Labrador, cited at least four articles that do not exist, under the names of researchers who denied them. KPMG published a report on customer experience in the age of AI with 45 citations. Five were correct. EY withdrew a study on fraud in loyalty programmes because 16 of its 27 sources were fabricated, misattributed or led to broken pages.
What stayed with me was not the fake citations. It was the line they closed with. In Australia, the department that commissioned the report said the substance stood and the recommendations did not change. In Canada, according to the provincial ministry, Deloitte said the same: it stood by its conclusions.
If the sources were false, what were the conclusions standing on?
The chain works like this. The government trusts the firm because of its name. The firm trusts its consultant, and the consultant trusts the tool. The tool writes something plausible, citations included. At the end, the firm's seal turns that text into an authoritative report.
The hallucination enters the chain with no authority at all and leaves it with plenty. It is what the guy on Facebook does, only with public money.
The mechanism is not the same at both ends, though. The guy on Facebook does not check because he wants to be right. With the consultant you do not need to assume any wish at all. It is enough that checking cost him time, he paid for that time, and the error, if anyone found it, would be paid for by the client. Two different roads ending in the same place.
And this is what interests me most. Nobody appointed AI as an authority. It has no title and no office. Its authority is borrowed: it summarises what science built up through replication and citation, it cites it, and in citing it keeps the prestige, even when the reference is invented. That is why the paper's title says perceived authority. It is not in the machine. It is in whoever reads it.
Verification has levels, and the first one is absurdly cheap. Does the cited document exist? A DOI takes a minute to look up. That step does not tell you whether the study is right, or even whether it says what the citation claims. It only tells you whether it exists. In the Big Four cases, nobody spent that minute.
There is a fix that looks obvious: have one AI review what another wrote. The problem is that the large models learned from almost the same books. A second opinion from someone who studied from the same textbook is not a second opinion. Whether that really happens, and how much, can be measured. The paper proposes how.
It proposes it because all of this is a thesis, not a fact. And a thesis claiming that people do not verify cannot ask to be believed. So before a single data point is collected, the paper writes down the conditions under which I would be wrong. If people verify just as much when the text says "an AI's answer" as when it says nothing, and adjust what they do to what is at stake, the thesis breaks. That result gets published too.
A confession, to finish. I wrote the paper with the tool the paper examines. In one of the review rounds, the AI checking the sources told me that a KPMG statement seemed to be misattributed to another firm. It sounded sure. It said so without having looked. When it finally looked, the quotation was there, word for word, in an article it had not opened.
It did not invent a source. It invented that the source did not exist.
I wrote it down in the paper's appendix, along with the other errors. It seemed only fair. If I cannot show where I failed, nobody has a reason to believe me where I got it right.