Confident report, no reproducible artifact
- Id
- irreproducible-confident-report
- Status
- Active
- Severity
- high
- Detection
- judge
- Evidence grade
- corroborated
- Languages
- en
- Added
- 2026-08-15
- Updated
- 2026-08-15
Currently signals low-effort writing.
What it is
A technical report with every section in the right place, describing something that does not happen. Steps to reproduce that reproduce nothing. A severity rating for a condition nobody can trigger.
Why it reads as machine-made
Report structure is a template and the artifact is not. The structural analogue of the fake citation: correct form, missing referent. The curl project closed its bug bounty programme in January 2026 under a flood of machine-written submissions, and the press account records that in the final week none of the reports described a real vulnerability. The cost of this tell falls entirely on reviewers, which is the asymmetry that makes it worth cataloguing: minutes to produce, hours to refute.
Specimens
Steps to reproduce: send a request with a malformed Host header. The parser dereferences a null pointer and the process exits. Impact: remote denial of service. Severity: high.
Steps to reproduce: on version 4.2.1 built with the flags in the attached log, run the attached script, which sends 200 requests with an empty Host header. The process exits on request 137 in three runs out of three. Core dump attached. On 4.1.9 it does not happen, so the change is somewhere between those two tags.
Versions and counts in this repair are invented for the specimen. The repair adds an artifact, a version boundary and a repeat count, which is what makes a report actionable.
How it is detected
- Rubric
- Task: decide whether the report contains anything a reviewer could run. Step 1. Check for an artifact. Score each present or absent: a script, payload or input file; a capture or log; a stack trace or crash dump; the exact version and build; the environment, including operating system and compiler or runtime; a repeat count across runs. Step 2. Check internal consistency. Does the described mechanism produce the described symptom? Does the severity rating follow from the described impact? Step 3. Check specificity. Are line numbers, function names and version boundaries given, and do they exist in the code as published? Escape hatches: reports under embargo that say so and offer the artifact privately; environment-specific issues where the reporter states the environment and the limits of what they can share; first-time reporters asking a question rather than asserting a finding; reports about closed-source systems where an artifact cannot be shared. Decision. FLAG when the report asserts a confirmed vulnerability or defect while missing an artifact, a version and an environment. Return the missing fields as a checklist the reporter can complete. Output: the artifact scorecard, the missing fields, and FLAG or PASS. A report with no artifact is unactionable, which is a statement about the report and not about the person who filed it.
Who writes this way legitimately
Novice security researchers file honest false positives constantly, and a programme that punishes them stops hearing from the people who later find real things. Bugs are also environment-specific in real cases: a race condition that appears on one scheduler, a fault that needs particular hardware, a failure that only shows under load nobody else can generate. Reporters working on closed systems cannot always share an artifact, and reporters in some jurisdictions have legal reasons to be careful about what they attach. The rubric therefore produces a checklist of missing fields rather than a verdict, because a report that becomes actionable after one more round was worth the round.
Model attribution
Undocumented per vendor. The reports in the documented case were submitted through a public programme and no tool was identified.
Status history
| Date | Status | Rationale |
|---|---|---|
| 2026-08-15 | Active | A widely reported programme closure in January 2026, covered by two independent outlets, with the maintainer describing a flood of machine-written reports. Rates circulating with that story could not be confirmed in the cited coverage and are not used here. |
Sources
- 01The Register, curl shutters bug bounty program to stop AI sloppressaccessed 2026-08-14
- 02BleepingComputer, curl ending bug bounty program after flood of AI slop reportspressaccessed 2026-08-14
CC BY 4.0 / The AI Tells Index, feedsquad.com/ai-tells