AI Hallucinations in Research:
Fabricated References, Invented Data and How to Catch Them
AI hallucinations in research are statements or references that look credible but cannot be confirmed against reliable evidence. Generative AI can still be useful for drafting, organizing and explaining research material. The safeguard is to verify each factual claim, citation and numerical result against the original source before publication (Walters & Wilder, 2023; van Dis et al., 2023).

A fabricated citation is not evidence that AI is useless; it is evidence that reference generation and reference verification are different tasks. Researchers can use AI to suggest a search direction, but the published bibliography must be built from works that have actually been located and read.
What can an AI hallucination look like?
| Type | What the researcher sees | What to check |
|---|---|---|
| Nonexistent reference | Plausible title, authors, journal and DOI | Does the item appear in a publisher or bibliographic database? |
| Real article with wrong details | Correct subject but altered authors or year | Exact metadata on the journal page |
| Unsupported evidence claim | A real reference attached to an unrelated conclusion | Does the original article actually support the sentence? |
| Invented result | A percentage, coefficient or sample size with no source | Original study table or original executed analysis |
| False interpretation | Correct statistical values followed by an unjustified causal conclusion | Design, estimand and alternative explanations |
The last two categories are risk examples, not measured hallucination frequencies. They are especially important because a citation can be real while its use is still misleading.
What does published research show about AI hallucinations in research?
Walters and Wilder (2023) assessed 636 references from 84 ChatGPT-produced texts across 42 topics. In that specific evaluation, 55% of GPT-3.5 references and 18% of GPT-4 references were fabricated. Some real citations also contained substantive bibliographic errors. These figures describe the models, prompts and task evaluated in 2023, not the performance of every model available today.
The practical lesson is narrower and more useful than “never use AI”: do not treat a generated reference list as a verified search result. Models, browsing connections and retrieval methods change, and performance must be assessed for the actual workflow.
How to verify AI-generated references

- Search the exact title in a recognized index or publisher site. If the title is unavailable, try the DOI, first author and a distinctive phrase.
- Open the publisher record. Match the title, author list, year, volume, pages or article number and DOI.
- Read the original abstract and relevant methods/results. Existence alone does not establish that the study supports your claim.
- Record evidence locations. Keep the page, table, figure or section that supports the article’s statement.
- Check quotations separately. A plausible quotation must be located word for word in the original.
- Recheck the bibliography before submission. References can be damaged by copying between citation managers, drafts and AI-generated text.
This procedure is an editorial recommendation grounded in the errors documented by Walters and Wilder (2023), not a journal-mandated checklist.
What about AI-generated data or tables?
A language model can construct a realistic-looking example dataset, useful for testing syntax or explaining a method. That is different from analyzing observed participant data. Label synthetic teaching material clearly, keep it separate from source records, and never insert invented values into a research results table.
If AI proposes a revised table, trace every number to an executed script, a verified extraction or the original dataset. For a statistical analysis, preserve the variable definitions, sample inclusion rules and outputs needed to reproduce the table (Sandve et al., 2013). Independent Statistical Review covers analyses prepared with AI assistance.
How can AI assist without becoming your evidence source?
A productive division of work is: use AI to propose search terms, checks, code, section outlines and explanations; use databases, primary studies and executed analyses to establish the claims. A researcher can ask AI to critique an argument, but that critique is a starting point for verification rather than an independent proof.
For sensitive data, avoid posting participant-level records to an unapproved tool simply to cross-check a table. An authorized local analysis can reproduce numerical output without transferring the dataset to an AI service.
Frequently asked questions
Does a DOI prove a reference is real?
No. Confirm that the DOI resolves to the same article, authors and journal cited. An AI system may attach a valid DOI to the wrong title.
Can AI verify its own citations?
It can organize an audit, but a second generated answer is not independent verification. Check publisher or indexed records directly.
Should I remove every AI-assisted sentence?
Not necessarily. Verify its factual content, apply your journal’s disclosure policy and retain author responsibility for the finished text (ICMJE, 2026).
Independent Statistical ReviewWant the numbers in your manuscript checked? A statistician reviews whether each reported value follows from a correctly specified analysis of your data.
Independent Statistical Review
Independent Statistical Review from OnlineSPSS is carried out in-house by PhD-level statisticians. The review checks method, assumptions, reproducibility and reporting, and your team keeps responsibility for the interpretation and conclusions.
References
- International Committee of Medical Journal Editors. (2026). Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals: Use of AI by authors. https://www.icmje.org/recommendations/browse/artificial-intelligence/ai-use-by-authors.html
- Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013). Ten simple rules for reproducible computational research. PLOS Computational Biology, 9(10), e1003285. https://doi.org/10.1371/journal.pcbi.1003285
- van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023). ChatGPT: Five priorities for research. Nature, 614, 224–226. https://doi.org/10.1038/d41586-023-00288-7
- Walters, W. H., & Wilder, E. I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13, Article 14045. https://doi.org/10.1038/s41598-023-41032-5
Last reviewed November 2026 by the OnlineSPSS Statistical Team. Worked examples use hypothetical values and are not client data.
NEXT STEP
Need this done for
your study?
Share your research in about a minute and receive a complimentary assessment the same day: scope, timeline and one fixed fee.
No Commitment Required
Strictly Confidential
Fixed Project Fee
In-House PhD-Level Statisticians
Same-Day Response
Serving Researchers Since 2014