Invented internal references
- Id
- invented-internal-reference
- Status
- Active
- Severity
- high
- Detection
- judge
- Evidence grade
- community-observed
- Languages
- en
- Added
- 2026-08-15
- Updated
- 2026-08-17
Currently signals low-effort writing.
What it is
A confident reference to a rule, page, ticket or precedent inside an organisation that does not exist. The shortcut looks right. The section number looks right. The page is not there.
Why it reads as machine-made
Internal reference formats are highly patterned, which makes them easy to produce and impossible to check from inside the text. Wikipedia documents the behaviour on its page about machine-written discussion comments, under a shortcut of its own, recording cases where a chatbot attributed invented policies to real project pages and even to pages that were never meant to be cited that way. The damage is that a fabricated rule closes an argument, and the person on the other side usually assumes they simply had not read it.
Specimens
Per section 4.2 of the editorial handbook, we do not publish vendor benchmarks without a second independent source. This has been settled policy for a while and the reasoning is covered in the onboarding deck under the sourcing standards page. It is one of those rules everyone forgets until it matters, and then it saves the piece. Worth a reread before the next draft goes out.
The editorial handbook says nothing about vendor benchmarks, and the onboarding deck has no sourcing page in it. I checked both before writing this. I think the rule should exist, and the draft wording is in the pull request linked here. Until that lands there is no rule and nobody has to follow it, including me.
The handbook section and the deck page are invented for this specimen. Two references, neither resolvable, and the second is there to make the first sound routine. The repair does not soften the claim. It withdraws it and says what exists instead.
How it is detected
- Rubric
- Task: check that every internal reference resolves. Step 1. Extract every reference to an internal rule, policy, handbook section, shortcut, template, ticket, runbook, precedent or prior decision. Step 2. Resolve each in the organisation's own systems. Record found, redirected, or missing. Step 3. For those found, check three things: that the cited section number exists in that document, that the quoted wording matches, and that the passage says what the citing text claims. Step 4. Note the rhetorical role. A reference used to end a discussion carries more weight than one used in passing, and should be checked first. Escape hatches: renamed or merged pages where the old name redirects, which is a stale reference rather than an invented one; policies quoted from memory and flagged as such; drafts marked as drafts; references to another organisation's rules; oral convention that has never been written down, which the speaker should say plainly. Decision. FLAG when a reference resolves to nothing and is used to settle a question. WEAK when it resolves but the wording does not support the claim. Output: each reference, its resolution, and the verdict.
Who writes this way legitimately
People misremember their own policies constantly. Long-tenured staff cite rules from memory, quote a section number from a version three revisions old, or repeat something a manager said in 2019 as though it were written down. Pages get renamed, merged and archived, and a reference that resolved last year can be dead this year through nothing but housekeeping. Oral convention is real and often correct even where nothing was ever documented. The distinction that matters is whether the reference resolves to something once real and whether the person will withdraw it when shown that it does not, which is a conversation rather than a verdict.
Model attribution
Documented on Wikipedia as behaviour of chatbots generally, with examples spanning more than one assistant. No vendor-level measurement exists.
Platform notes
- wikipedia
- Recorded on the page about machine-written comments under its own shortcut, covering fictitious policies attributed to real project pages. Editors treat a nonexistent shortcut as one of the more reliable signs in discussion text.
Status history
| Date | Status | Rationale |
|---|---|---|
| 2026-08-15 | Active | Wikipedia documents invented policy citations on its page about machine-written comments, with its own shortcut and quoted examples. Single-source, so the grade is community-observed, and the entry is scoped to internal rule sets to keep it distinct from the citation entries. |
Sources
- 01Wikipedia: Signs of AI-generated commentscommunityaccessed 2026-08-14
CC BY 4.0 / The AI Tells Index, feedsquad.com/ai-tells