Edition 26: Check an AI citation against the paper it links to

A graduate student asks an assistant for a reference, clicks the supplied DOI, and lands on a real journal page. That feels like verification. It is only the first clue.

A DOI, or digital object identifier, is a persistent identifier for a publication or another research object. A real DOI can be attached to the wrong title, author or claim in a draft. The link works while the reference is still wrong.

Here is the task: turn a suspicious reference into a citation-review note that another person can inspect. You need a browser and a document you can edit. No paid model, download or Python installation is needed for the main route.

This table is an original teaching example. The wrong title is a seeded mistake, not an observed model response. The retrieved identity comes from an actual Crossref API request on October 27, 2026.

The useful result is more modest than a green “verified” badge: a corrected reference and an explicit record of what still needs reading. That distinction belongs in any assistant that helps prepare a literature review, teaching handout or evidence-based report.

A working link can hide two different mistakes

There are three separate questions. Does an identifiable work exist? Does its metadata match the reference? Does the work support the sentence beside the citation? Metadata means descriptive fields such as title, authors, journal and publication date.

A lookup can help with the first two. It cannot answer the third just by returning a record.

This is an established research problem. Walters and Wilder’s 2023 study examined references in literature reviews generated by GPT-3.5 and GPT-4 and distinguished fabricated works from errors in references to real works. Those historical results do not measure a current assistant.

The GhostCite preprint, revised May 14, 2026, reports a separate investigation of citation validity, including generated references and published papers. It is a preprint with its own selection and verification methods, not a forecast of your next draft’s error rate. Together, these studies justify checking the identity of a reference. Neither proves that this article’s format will help every reader.

Start with the reference, not another confident answer

For one or two references, ordinary searching and a small review note are often sufficient. You do not need an autonomous agent to do this work.

An assistant may help when a report has many references to organize, provided it returns lookup records and unresolved cases rather than silently choosing replacements. A fixed lookup step retrieves metadata; a model can propose what to investigate next. A person remains responsible for accepting the reference and its use.

Use this complete task with an assistant that already has public web access, or follow it yourself:

Check DOI 10.1038/s41598-023-41032-5 against this supplied title: “AI citations are always reliable.” Retrieve the publication’s title, authors, year and journal from a public bibliographic record. Link the record. Compare each field rather than accepting a working URL as a match. Do not invent a substitute source. Return a corrected reference, the mismatch you found, and a separate claim-support status. Mark claim support NOT CHECKED unless you actually read and identify the relevant passage. If retrieval fails, report retrieval failure rather than declaring the work fake.

Only supply public bibliographic information to a public lookup. An unpublished manuscript or confidential client report should not be pasted into a service merely to find a citation.

1. Retrieve the identity independently

Open the Crossref record for this DOI. It returns JSON, a structured text format. Look inside its message object for title, author, published, container-title, volume and article-number.

The executed request returned:

• Title: Fabrication and errors in the bibliographic citations generated by ChatGPT.
• Authors: William H. Walters and Esther Isabelle Wilder.
• Published date: September 7, 2023.
• Journal: Scientific Reports.
• Volume and article number: 13 and 14045.

Crossref documents that its public API exposes deposited scholarly metadata without requiring sign-up. This was a read-only request without credentials. It did not retrieve or evaluate the full paper.

The title mismatch is visible immediately. Repair it from the record, keeping a note of the original mistake. Do not ask the same assistant to reassure you that its earlier reference was correct.

A missing Crossref result is not conclusive evidence of fabrication. Other registration agencies exist, records can be incomplete, and services can fail. Search the title and authors through a library catalogue or publisher, and ask a librarian when the identity remains uncertain.

2. Keep the reference and the claim in separate boxes

Copy this editable result into your document:

Reference: Walters, W. H., and Wilder, E. I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13, 14045. https://doi.org/10.1038/s41598-023-41032-5
Identity check: Crossref title, author surnames, year and journal agree with the corrected reference. Checked October 27, 2026.
Correction: The supplied title was replaced.
Proposed use: Background on fabricated references and bibliographic errors in generated literature reviews.
Claim-support status: NOT CHECKED in this exercise’s metadata lookup.
Next action: Read the relevant passage, record its location and narrow the report’s sentence to what it supports.

Now consider a second seeded mistake: “This paper proves that today’s assistants never invent citations.” Correcting the reference would not rescue that sentence. The paper’s topic, historical model versions and actual findings must be examined. Remove or rewrite the sentence after reading, rather than attaching a better-looking citation.

For your own work, replace the DOI, supplied fields and proposed use. Add the exact claim you intend to make and a page, section or paragraph location after reading. Keep inaccessible full text and unresolved identity visible.

Zotero helps collect a record, not approve an argument

If you already use Zotero, its official Adding Items documentation describes adding items by identifier, including DOI, and saving records from publication pages with its browser Connector. That can reduce retyping.

This capability was inspected in documentation; Zotero was not installed or executed for this article. Its desktop setup, Connector permissions and destination library need checking before use. Imported fields still deserve review, and imported metadata does not certify that a paper supports your argument.

For a single reference, the browser-and-note route above avoids that setup. A reference manager becomes more useful when you need to organize and reuse a library.

Optional: reproduce the identity boundary offline

This small Python exercise records a metadata comparison. It does not search the internet, read papers, run a model or automatically evaluate claim support. It uses the minimal fields from the live lookup, followed by deliberately altered cases.

With Python 3.12 and a text editor, create an empty folder. Save the following complete file as check.py. No other input file or package is required.

from pathlib import Path
import csv
import io

# Minimal metadata transcribed from the live Crossref lookup on 2026-10-27.
record = {
    "doi": "10.1038/s41598-023-41032-5",
    "title": "Fabrication and errors in the bibliographic citations generated by ChatGPT",
    "authors": "Walters; Wilder",
    "year": 2023,
}
def identity(candidate):
    # Deliberately strict: a mismatch needs review, not automatic fuzzy repair.
    return all(type(candidate.get(k)) is type(v) and candidate[k] == v
               for k, v in record.items())

valid = dict(record)
wrong_title = dict(record, title="AI citations are always reliable")
wrong_year = dict(record, year=2024)
missing_doi = {k: v for k, v in record.items() if k != "doi"}
cases = [("matched", valid), ("wrong_title", wrong_title),
         ("wrong_year", wrong_year), ("missing_doi", missing_doi)]
assert identity(valid)
assert not any(identity(c) for _, c in cases[1:])
repaired = dict(wrong_title, title=record["title"])
assert identity(repaired)
output = Path("citation-review.csv")
if output.exists():
    raise SystemExit("Keep the existing review; run in a clean folder.")
buffer = io.StringIO(newline="")
writer = csv.writer(buffer, lineterminator="\n")
writer.writerow(["case", "identity", "claim_support"])
for name, candidate in cases:
    status = "MATCH" if identity(candidate) else "REVIEW"
    writer.writerow([name, status, "NOT_CHECKED"])
    print(f"{name}: {status}; support=NOT_CHECKED")
output.write_text(buffer.getvalue(), encoding="utf-8")
print("repair: MATCH; support=NOT_CHECKED")
print("saved=citation-review.csv")

Run python -I check.py from that folder. This was checked with Python 3.12.14 in a disposable directory with a credential-free environment.

Expected complete stdout:

matched: MATCH; support=NOT_CHECKED
wrong_title: REVIEW; support=NOT_CHECKED
wrong_year: REVIEW; support=NOT_CHECKED
missing_doi: REVIEW; support=NOT_CHECKED
repair: MATCH; support=NOT_CHECKED
saved=citation-review.csv

Open citation-review.csv in a spreadsheet or text editor. It contains four data rows: one MATCH and three REVIEW results. Every claim-support cell remains NOT_CHECKED. The repaired title passes identity comparison while still requiring reading.

Strict comparison intentionally sends punctuation differences and alternative author formatting for review too. It is a teaching boundary, not a general bibliographic matching algorithm. Adding fuzzy matching without inspecting false matches could hide the error this exercise exposes.

Troubleshooting: If the output file already exists, preserve it and use another empty folder. If Python is unavailable, use the manual note. If a live DOI request times out or returns an error, retain the failure and try a library or publisher route; do not turn an outage into a verdict about the paper.

Finish with a check someone else can repeat

Before using your reference, another reader should be able to open the linked record, find the corrected identity and distinguish it from your proposed claim. In the optional exercise, independently count the four CSV rows and confirm that a matching identity never changes NOT_CHECKED into an approval.

The technical execution checked stdout, CSV contents, three deliberate failures, the repair and refusal to overwrite an existing result. The author performed the technical and editorial passes in the same AI authoring session. There was no independent novice trial or current-model benchmark. Review time, maintenance effort and provider costs were not measured, so no savings claim follows.

For your next experiment, choose one real reference from a report you are preparing. Retain the original draft, lookup evidence, actual source passage and repaired sentence. Record the review and repair effort as well as whether the final sentence is supported. That turns a plausible bibliography into a traceable piece of research work.

This extends the journal’s source-based brief lesson to scholarly references. The start guide shows where research, tools and human review fit together.