Detectability of LLM-Generated Text Remains Ill-Defined, Benchmarks Fail to Address Real-World Conditions

The increasing prevalence of large language models (LLMs) has sparked considerable effort into developing methods to identify text they produce, but a fundamental problem underlies this pursuit, as researchers Mingmeng Geng and Thierry Poibeau from École Normale Supérieure, Université Paris Sciences et Lettres, demonstrate. Their work reveals a critical gap in current approaches, stemming from the lack of a clear definition of what actually constitutes “LLM-generated text”. The team highlights how human editing of LLM outputs, combined with the ways these models subtly influence writing styles, blurs the boundary between machine and human authorship. This research is significant because it shows that existing benchmarks often misrepresent detector performance, limiting their usefulness in real-world applications and demanding a more nuanced understanding of their capabilities.

Currently, identifying text generated by large language models presents significant challenges. Differences in application and the diversity of available models increase the difficulty of accurate detection. What detectors typically target represents only a portion of the text large language models are capable of producing. Human editing of model outputs, combined with the subtle ways these models influence users, obscures the distinction between machine- and human-written text. Consequently, detectors maintain utility only under specific, limited circumstances.

Detecting and Characterising Language Model Text

Researchers are actively developing methods to identify text generated by large language models, yet establishing a precise definition of “LLM-generated text” remains a significant challenge. The study highlights that current detection efforts often target only a subset of the text LLMs can produce, overlooking the influence of human editing and the subtle ways LLMs shape user writing. To address this, scientists have pioneered a range of detection techniques, beginning with tools like GLTR and Grover, which emerged before the widespread release of ChatGPT. These early systems, employing models such as GPT-2 and BERT, aimed to distinguish machine-generated text from human writing.

The field rapidly expanded with the introduction of DetectGPT, Fast-DetectGPT, DetectLLM, and numerous other detectors proposed between 2023 and 2025. These methods utilize diverse approaches, including supervised learning, zero-shot classification, retrieval-based analysis, and watermarking techniques. Researchers categorize these detectors based on criteria such as the underlying methodology and the type of content analyzed, with specialized tools developed for specific domains like tweets, news articles, and Wikipedia entries. Despite this progress, the study emphasizes the lack of universal benchmarks and consistent application scenarios, hindering meaningful comparisons between detectors. Scientists are actively investigating the limits of detectability, with some arguing that accurate differentiation is achievable as long as human and machine text distributions remain distinct, while others contend that perfect detection is mathematically impossible. This ongoing debate underscores the complex interplay between human intervention, evolving LLM capabilities, and the challenges of reliably identifying machine-generated text in real-world contexts.

Defining LLM Text and Detection Limits

This research rigorously examines the fundamental challenge of defining “LLM-generated text” and its implications for detection accuracy. Scientists demonstrate that current approaches often target only a subset of the text large language models (LLMs) are capable of producing, leading to limited and potentially misleading results. The work highlights that the detection target is frequently defined too broadly, encompassing text produced in various ways, such as paraphrasing, translation, or simple prompt responses, rather than focusing on a precise definition of what constitutes LLM authorship. Researchers establish that evaluating text based solely on its final output creates considerable overlap between LLM-generated and human-written content, making accurate detection increasingly difficult.

The study reveals that many detectors are trained on limited datasets, representing only a fraction of the possibilities LLMs can generate, and consequently, their detection capabilities are constrained. The team’s investigation extends to real-world applications, noting the increasing integration of LLM-generated text into diverse areas, including student essays, spoken language, and online content. This pervasive integration further complicates the task of distinguishing between human and machine authorship. Scientists conclude that, given these challenges, accurately detecting LLM-generated text in many practical cases is not currently possible, emphasizing the need for a more nuanced understanding of the problem and a re-evaluation of current detection methodologies.

Detecting LLM Text, A Growing Challenge

This research demonstrates the complex challenges inherent in detecting text generated by large language models. The team highlights a fundamental issue: a lack of clear definition regarding what constitutes “LLM-generated text” given the increasing integration of these models into human writing processes. Their work reveals that simple detection metrics can be misleading, as even text initially produced by a language model, and then refined through further LLM processing, can be misclassified by current detection tools. Results show that detectors often perform inconsistently, sometimes identifying human-edited LLM outputs as less machine-generated than the original LLM draft.

The study underscores that numerical outputs from detectors should be interpreted cautiously, serving as reference points rather than definitive indicators of origin. The team acknowledges the current lack of understanding surrounding the stylistic mechanisms of language models and the limited interpretability of existing detectors, hindering transparent explanations for the public. Future work, they suggest, should focus on developing more nuanced evaluation methods that account for the collaborative nature of writing with LLMs and address the underlying causes of detection errors.