---
type: source
id: lu-2023-fake2m
title: 'Seeing is not always believing: Benchmarking human and model perception of AI-generated images'
authors:
- Zeyu Lu
- Di Huang
- Lei Bai
- Jingjing Qu
- Chengyue Wu
- Xihui Liu
- Wanli Ouyang
year: 2023
venue: NeurIPS 2023 Datasets and Benchmarks Track
volume: null
pages: null
doi: null
url: https://arxiv.org/abs/2304.13023
arxiv_id: '2304.13023'
peer_review: peer-reviewed-conference
evidence_tier: B
design: Human perception benchmark (HPBench) on AI-generated vs real images; model benchmark (MPBench) on the Fake2M dataset
modality:
- image-general
stimuli: Text-to-image diffusion outputs vs real photos
n_participants: null
n_note: not stated in abstract
topics:
- human-detection
- text-to-image
- human-vs-machine
summary: Humans misclassified 38.7% of images; the best model failed on 13% under the same conditions.
results:
- metric: Human misclassification rate (HPBench)
  value: 38.7%
  note: ≈61.3% accuracy
  source_location: abstract
- metric: Best model failure rate (MPBench)
  value: 13%
  note: ≈87% accuracy
  source_location: abstract
moderators: []
limitations:
- Not face-specific
- Venue (NeurIPS D&B) from compiler knowledge
verification:
  status: verified-abstract
  confidence: high
  checked: 2026-10-07
  via: arXiv 2304.13023 abstract
pmid: null
pmcid: null
openalex_id: null
oa_url: null
cited_in:
- MR-0001
updated: '2026-10-07'
---

# Seeing is not always believing: Benchmarking human and model perception of AI-generated images

> **Source record** `lu-2023-fake2m` · 2023 · NeurIPS 2023 Datasets and Benchmarks Track  
> **Evidence tier:** B (peer-reviewed-conference) · **Verification:** verified-abstract (compiler confidence: high)  
> **Bottom line:** Humans misclassified 38.7% of images; the best model failed on 13% under the same conditions.

## Citation {#citation}

Zeyu Lu, Di Huang, Lei Bai, Jingjing Qu, Chengyue Wu, Xihui Liu, Wanli Ouyang (2023). Seeing is not always believing: Benchmarking human and model perception of AI-generated images. *NeurIPS 2023 Datasets and Benchmarks Track*. https://arxiv.org/abs/2304.13023

Verification entry (identifiers, resolvers, claims with source locations): [/data/citations.json#lu-2023-fake2m](/data/citations.json#lu-2023-fake2m)

## Study design {#design}

| Field | Value |
|---|---|
| Design | Human perception benchmark (HPBench) on AI-generated vs real images; model benchmark (MPBench) on the Fake2M dataset |
| Modality | image-general |
| Stimuli | Text-to-image diffusion outputs vs real photos |
| Participants (N) | not recorded |
| N detail | not stated in abstract |
| Topics | human-detection, text-to-image, human-vs-machine |

## Results {#results}

| Metric | Value | Note |
|---|---|---|
| Human misclassification rate (HPBench) | 38.7% | ≈61.3% accuracy |
| Best model failure rate (MPBench) | 13% | ≈87% accuracy |

## Limitations {#limitations}

- Not face-specific
- Venue (NeurIPS D&B) from compiler knowledge

## Cited in {#cited-in}

- [MR-0001](/metareports/human-detection-accuracy.md): Human Detection of Deepfakes: Accuracy, Bias, Confidence, and Training

## Verification {#verification}

| Field | Value |
|---|---|
| Status | `verified-abstract` |
| Compiler confidence | high |
| Checked on | 2026-10-07 |
| Via | arXiv 2304.13023 abstract |

Values on this page have not all been checked against the full text. Confirm them against the primary source before high-stakes use.