X-Aigd Benchmark Achieves Fine-Grained Detection of -Generated Image Artifacts

Researchers are tackling the challenge of reliably identifying images created by artificial intelligence, a growing concern as these become increasingly realistic. Yao Xiao, Weiyan Chen, and Jiahao Chen, from Sun Yat-sen University and Xi’an Jiaotong University, alongside Zijie Cao, Weijian Deng, and Binbin Yang et al., have introduced a new benchmark, X-AIGD, to move beyond simple ‘real or fake’ judgements. This benchmark uniquely provides detailed, pixel-level annotations of the subtle flaws , or perceptual artifacts , within AI-generated images, ranging from basic distortions to more complex semantic and cognitive inconsistencies. By pinpointing where and why a detector flags an image as artificial, this work exposes a critical weakness in current detection methods , their limited focus on these actual artifacts , and paves the way for more transparent and trustworthy AI detection systems.

X-AIGD benchmark for detailed AIGI artifact analysis

Current AIGI detection methods often rely on simple binary classification, lacking convincing evidence to support their decisions, and existing benchmarks fall short in comprehensively covering the diversity of synthetic image artifacts with detailed annotations. These comprehensive annotations enable fine-grained interpretability evaluation and offer deeper insights into the decision-making processes of AIGI detection models. The research team meticulously constructed X-AIGD to include a hierarchical taxonomy of perceptual artifacts, categorising them into three levels: low-level distortions like unnatural textures and warped edges, high-level semantic errors concerning object structure, and cognitive-level counterfactuals representing violations of commonsense or physical laws. This structured approach captures a wider range of inconsistencies than previous datasets, offering a more thorough evaluation of model capabilities.
To ensure data quality and relevance, the team generated diverse fake images from real image captions using advanced models such as FLUX and Stable Diffusion 3.5, maintaining semantic alignment and distribution consistency between real and synthetic pairs. The resulting X-AIGD dataset comprises paired real and fake images with pixel-level annotations and systematic artifact categorization, providing a robust resource for interpretable AIGI detection research. Despite their ability to learn complex features, these detectors largely bypass the very cues humans use to identify synthetic images, highlighting a significant opportunity for improvement. While training detectors to identify specific artifacts is possible, the study shows they still heavily depend on uninterpretable features for their judgments.

This breakthrough establishes a clear path towards more transparent and reliable AIGI detection systems, with potential applications ranging from combating misinformation to ensuring the authenticity of visual content. The availability of the X-AIGD dataset and associated code at https://github. com/Coxy7/X-AIGD will empower researchers to develop and evaluate new methods that prioritize interpretable, artifact-based reasoning, ultimately fostering greater trust in AI-generated imagery and its applications. The work opens exciting avenues for future research into the intersection of computer vision, explainable AI, and the detection of increasingly realistic synthetic media.

X-AIGD Dataset Creation and Pixel Annotations are now

Scientists developed the X-AIGD benchmark, a fine-grained dataset for explainable AI-Generated Image Detection, to address limitations in existing approaches. Researchers meticulously curated a dataset comprising paired real and fake images, generated from captions using advanced models like FLUX and Stable Diffusion 3.5, ensuring semantic alignment and consistent distributions between the pairs. The study collected a total of 7,892 image pairs, each accompanied by precise pixel-level annotations delineating the location and category of perceptual artifacts, providing a robust foundation for analysis. To construct X-AIGD, the team engineered a three-tiered taxonomy of perceptual artifacts, categorising them by complexity and observability.

Low-level distortions, such as edge misalignments and unnatural textures, were identified and annotated, alongside high-level semantic errors affecting object integrity and arrangement. Crucially, the research extended to cognitive-level counterfactuals, capturing violations of commonsense and physical laws, demanding a deeper understanding of image realism. Experiments employed human annotators to meticulously label these artifacts within each image, ensuring high-quality, spatially-grounded annotations for reliable interpretability assessment. Scientists trained detectors to specifically identify these artifacts, but found they still heavily depended on uninterpretable features for overall judgment.

This prompted the team to pioneer a method of explicitly aligning model attention with artifact regions, demonstrating increased interpretability and generalization capabilities in detection performance. The approach enables a more transparent decision-making process, allowing researchers to understand why a model classifies an image as fake, rather than simply that it is fake. Furthermore, the study meticulously analysed the performance of these models, quantifying their reliance on different artifact categories and assessing the impact of explicit artifact attention. Data and code are publicly available, fostering reproducibility and further research within the community, accessible at https://github. com/Coxy7/X-AIGD.

X-AIGD reveals detectors ignore perceptual artifacts

Scientists have developed a new benchmark, named X-AIGD, to address limitations in current AI-generated image (AIGI) detection methods. Researchers measured the performance of models like CNNSpot, Gram-Net, FatFormer, DRCT-CLIP, and CoDE, evaluating their ability to leverage perceptual cues for detection. Data shows that detector accuracy does not significantly correlate with image fidelity, as measured by NIQE, MDFS, and Perceptual Artifact Ratio (PAR). Interestingly, models failed to consistently perform better on AIGIs with visible artifacts (PAR 0) compared to those without (PAR = 0), indicating limited utilisation of perceptual cues.

Quantitative evaluation, employing Grad-CAM and Relevance Map techniques, indicated weak alignment between model interpretations and human perception, with a Category-Agnostic PAD score remaining low across tested models. Qualitative analysis revealed that FatFormer concentrates on background regions with smooth textures, while DRCT-CLIP exhibits a scattered activation pattern, demonstrating a lack of consistent attention to perceptual artifacts. Experiments demonstrated that training models on artifact annotations achieves a non-trivial performance of 27.2 IoU on Category-Agnostic PAD, but transfer learning and multi-task learning yield at most marginal improvements in Authenticity Judgment (AJ). Table 2 presents a detailed comparison, showing that the multi-task learning approach achieves an F1 score of 92.3 on AJ and 42.8 on PAD.

Further fine-grained evaluation on perceptual artifact detection revealed that models perform better on low-level artifacts, achieving a PixP of 17.7 for textures and 32.8 for edges & shapes, but struggle with high-level ones. Specifically, the multi-task model achieved a PixP of 14.9 for textures and 29.8 for edges & shapes, while performance on cognitive-level artifacts remained low at 3.4 PixP. The team measured performance using metrics like Precision, Recall, F1 score, IoU, PixP, and PixR, providing a comprehensive assessment of both authenticity judgment and perceptual artifact detection. The data and code are publicly available, enabling further research and development in this critical area.

X-AIGD Reveals Detector Reliance on Obscure Features, hindering

Scientists have introduced a new fine-grained benchmark, named X-AIGD, to improve the detection of AI-generated images (AIGI). The research addresses a key limitation in current AIGI detection methods, which often lack transparency and detailed justification for their classifications. While detectors can be trained to identify specific artifacts, they continue to depend significantly on these obscure characteristics. Researchers found that explicitly modelling attention towards artifact regions can enhance both the interpretability and generalisation capabilities of detectors. The authors acknowledge a consistent imbalance between precision and recall in their findings, mirroring a previously identified challenge where detectors often misclassify real images as fake.

These findings underscore the importance of developing artifact-aware detection strategies for AIGI. By establishing a new standard for interpretability evaluation, X-AIGD aims to stimulate progress in creating robust and explainable AIGI detection methods. Future work could explore methods to better align detector attention with human-interpretable cues and further improve generalisation across diverse datasets. The study involved careful ethical considerations regarding data annotation and usage of publicly available datasets, including filtering for inappropriate content and ensuring compliance with licensing agreements.