Seeing is not always believing: Benchmarking human and model perception of AI-generated images
Source record
lu-2023-fake2m· 2023 · NeurIPS 2023 Datasets and Benchmarks Track
Evidence tier: B (peer-reviewed-conference) · Verification: verified-abstract (compiler confidence: high)
Bottom line: Humans misclassified 38.7% of images; the best model failed on 13% under the same conditions.
Citation
Zeyu Lu, Di Huang, Lei Bai, Jingjing Qu, Chengyue Wu, Xihui Liu, Wanli Ouyang (2023). Seeing is not always believing: Benchmarking human and model perception of AI-generated images. NeurIPS 2023 Datasets and Benchmarks Track. https://arxiv.org/abs/2304.13023
Verification entry (identifiers, resolvers, claims with source locations): /data/citations.json#lu-2023-fake2m
Study design
| Field | Value |
|---|---|
| Design | Human perception benchmark (HPBench) on AI-generated vs real images; model benchmark (MPBench) on the Fake2M dataset |
| Modality | image-general |
| Stimuli | Text-to-image diffusion outputs vs real photos |
| Participants (N) | not recorded |
| N detail | not stated in abstract |
| Topics | human-detection, text-to-image, human-vs-machine |
Results
| Metric | Value | Note |
|---|---|---|
| Human misclassification rate (HPBench) | 38.7% | ≈61.3% accuracy |
| Best model failure rate (MPBench) | 13% | ≈87% accuracy |
Limitations
- Not face-specific
- Venue (NeurIPS D&B) from compiler knowledge
Cited in
- MR-0001: Human Detection of Deepfakes: Accuracy, Bias, Confidence, and Training
Verification
| Field | Value |
|---|---|
| Status | verified-abstract |
| Compiler confidence | high |
| Checked on | 2026-10-07 |
| Via | arXiv 2304.13023 abstract |
Values on this page have not all been checked against the full text. Confirm them against the primary source before high-stakes use.