arbiter
JournalsPricingAccount

Do humans identify AI-generated text better than machines? Evidence based on excerpts from German theses

Alexandra Fiedler & Jörg Döpke

International Review of Economics Education2025https://doi.org/10.1016/j.iree.2025.100321article
ABDC C
Weight
0.57

What the paper says

We investigate whether human experts can identify AI-generated academic texts more accurately than current machine-based detectors. Conducted as a survey experiment at a German university of applied sciences, 63 lecturers in engineering, economics, and social sciences were asked to evaluate short excerpts (200–300 words) from both human-generated and AI-generated texts. These texts varied by discipline and writing level (student vs. professional) with the AI-generated content. The results show that both human evaluators and AI detectors correctly identified AI-generated texts only slightly better than chance, with humans achieving a recognition rate of 57 % for AI texts and 64 % for human-generated texts. There was no statistically significant difference between human and machine performance. Notably, professional-level AI texts were the most difficult to identify, with less than 20 % of respondents correctly classifying them. Regression analyses suggest that prior teaching experience slightly improves recognition accuracy, while subjective judgments of text quality were not influenced by actual or presumed authorship. These findings suggest that current written examination practices are increasingly vulnerable to undetected AI use. Both human judgment and existing AI detectors show high error rates, especially for high-quality AI-generated content. We conclude that a reconsideration of traditional assessment formats in academia is warranted. • A survey of 63 lecturers revealed that only half of the AI-generated texts could be recognized as such. • Humans recognize AI texts slightly better than AI detectors. • The higher the level of AI-generated texts, the more difficult it is to distinguish them from human texts. • Human assessment of text quality does not depend on whether the text is actually from an AI.

11 citations

Open paper page →

Cite this paper

https://doi.org/https://doi.org/10.1016/j.iree.2025.100321

Or copy a formatted citation

@article{alexandra2025,
  title        = {{Do humans identify AI-generated text better than machines? Evidence based on excerpts from German theses}},
  author       = {Alexandra Fiedler & Jörg Döpke},
  journal      = {International Review of Economics Education},
  year         = {2025},
  doi          = {https://doi.org/https://doi.org/10.1016/j.iree.2025.100321},
}

Paste directly into BibTeX, Zotero, or your reference manager.

Flag this paper

Do humans identify AI-generated text better than machines? Evidence based on excerpts from German theses

Flags are reviewed by the Arbiter methodology team within 5 business days.

Evidence weight

0.57

Balanced mode · F 0.40 / M 0.15 / V 0.05 / R 0.40

F · citation impact0.57 × 0.4 = 0.23
M · momentum0.78 × 0.15 = 0.12
V · venue signal0.50 × 0.05 = 0.03
R · text relevance †0.50 × 0.4 = 0.20

† Text relevance is estimated at 0.50 on the detail page — for your query’s actual relevance score, open this paper from a search result.