Adam Zachary Wasserman › Papers
The Off-arXiv Blind Spot
A Cross-Vendor Measurement of AI/ML Preprints arXiv Does Not Host
Adam Zachary Wasserman, Open Honest Foundation. Preprint, 10 July 2026.
Abstract
arXiv is the default preprint server for machine learning, but by its own admission it has been struggling with a dramatic increase in submissions, in part because of AI-assisted writing. Current moderation removes an unknown quantity of submitted work, and deposits that then live only on other repositories form a blind spot the field does not see. We motivate the problem with a documented rejection in which arXiv declined a submission and offered, as the sole route to reconsideration, prior publication in a conventional peer-reviewed journal, a requirement in tension with the definition of a preprint.
We then set that anecdote aside and measure the blind spot directly. Over a 90-day window we assembled 1,526 recent AI/ML deposits on Preprints.org and Zenodo, drew a reproducible random sample of 350, confirmed each as absent from arXiv, and scored four quality criteria (real research, broad relevance, testable, novel) using five large language models spanning four independent vendors, with inter-rater reliability reported. Cross-posting to arXiv was negligible (0.3%), so the off-arXiv corpus is effectively the entire blind spot.
About 7% of off-arXiv deposits clear all four criteria, but a majority are genuine checkable research: an estimated 66.5% (95% CI 61.4 to 71.2) are real and testable, and roughly half of the whole pile is real, testable, and novel work that fails only on breadth of interest, a curation preference derived from the study's origins rather than a validity judgment. A parallel sample of arXiv's own AI/ML papers scores far higher (96% admissible, no zero-of-four junk), which shows that arXiv's curation raises average quality and validates the instrument, since the same criteria cleanly separate a curated corpus from an uncurated one. Off-arXiv status is therefore not quality-neutral. Even so, a majority remains admissible, and a preprint gate which filters on provenance rather than on testability discards a large volume of checkable work.
Identifiers
- DOI (Zenodo, canonical): 10.5281/zenodo.21286250
- ORCID: 0009-0002-8865-6583
- Licence: CC BY 4.0
Cite as
Wasserman, A. Z. (2026). The Off-arXiv Blind Spot: A Cross-Vendor Measurement of AI/ML Preprints arXiv Does Not Host. Open Honest Foundation. https://doi.org/10.5281/zenodo.21286250