---
type: source
id: mai-2023-speech-deepfakes
title: 'Warning: Humans cannot reliably detect speech deepfakes'
authors:
- Kimberly T. Mai
- Sergi Bray
- Toby Davies
- Lewis D. Griffin
year: 2023
venue: PLOS ONE
volume: 18(8)
pages: e0285333
doi: 10.1371/journal.pone.0285333
peer_review: peer-reviewed-journal
evidence_tier: B
design: Online listening experiment (Prolific); unary (single clip) and binary (pick the fake from a pair) tasks; with vs without familiarisation (5 example deepfakes); English and Mandarin
modality:
- audio
stimuli: VITS text-to-speech deepfakes vs genuine speech (LJSpeech; Chinese Standard Mandarin Speech Corpus); 50 sentences per language, 100 clips, 2-11 s
n_participants: 529
n_note: ''
topics:
- human-detection
- voice
- training
- human-vs-machine
summary: Listeners were 70.35% accurate on single clips (73% hit rate on fakes); familiarisation added only ≈3.8 points; crowd-aggregated judgments rivalled in-domain detectors, while out-of-domain detectors failed.
results:
- metric: Unary accuracy (single clip)
  value: 70.35%
  note: deepfakes identified 73%; genuine 67.78%
  source_location: null
- metric: Binary accuracy (pick fake from pair)
  value: 85.59%
  note: ''
  source_location: null
- metric: Familiarisation effect
  value: +3.84 points (p=0.001)
  note: unary +3.76, binary +3.85
  source_location: null
- metric: Confidence-weighted AUROC
  value: 73.83% → 75.54% with familiarisation
  note: ''
  source_location: null
- metric: Crowd-aggregated AUROC
  value: 95.51% (94.04% with familiarisation)
  note: comparable to in-domain detectors
  source_location: null
- metric: Automated detectors AUROC
  value: in-domain 100%; out-of-domain 25.31%
  note: out-of-domain flagged genuine speech as fake
  source_location: null
- metric: Mandarin vs English listeners
  value: +1.79 points, n.s. (p=0.202)
  note: ''
  source_location: null
moderators:
- Reference audio of the same sentence helped significantly (p<0.001) but is rarely available in practice.
- Extra time and repeated listening did little to improve performance.
limitations:
- Older TTS (VITS), not current voice cloning
- Short clips
- Participants knew fakes were present
verification:
  status: verified-fulltext
  confidence: high
  checked: 2026-10-07
  via: journals.plos.org article page
url: null
pmid: null
pmcid: null
arxiv_id: null
openalex_id: null
oa_url: null
cited_in:
- MR-0001
- MR-0003
updated: '2026-10-07'
---

# Warning: Humans cannot reliably detect speech deepfakes

> **Source record** `mai-2023-speech-deepfakes` · 2023 · PLOS ONE  
> **Evidence tier:** B (peer-reviewed-journal) · **Verification:** verified-fulltext  
> **Bottom line:** Listeners were 70.35% accurate on single clips (73% hit rate on fakes); familiarisation added only ≈3.8 points; crowd-aggregated judgments rivalled in-domain detectors, while out-of-domain detectors failed.

## Citation {#citation}

Kimberly T. Mai, Sergi Bray, Toby Davies, Lewis D. Griffin (2023). Warning: Humans cannot reliably detect speech deepfakes. *PLOS ONE*, 18(8), e0285333. https://doi.org/10.1371/journal.pone.0285333

Verification entry (identifiers, resolvers, claims with source locations): [/data/citations.json#mai-2023-speech-deepfakes](/data/citations.json#mai-2023-speech-deepfakes)

## Study design {#design}

| Field | Value |
|---|---|
| Design | Online listening experiment (Prolific); unary (single clip) and binary (pick the fake from a pair) tasks; with vs without familiarisation (5 example deepfakes); English and Mandarin |
| Modality | audio |
| Stimuli | VITS text-to-speech deepfakes vs genuine speech (LJSpeech; Chinese Standard Mandarin Speech Corpus); 50 sentences per language, 100 clips, 2-11 s |
| Participants (N) | 529 |
| Topics | human-detection, voice, training, human-vs-machine |

## Results {#results}

| Metric | Value | Note |
|---|---|---|
| Unary accuracy (single clip) | 70.35% | deepfakes identified 73%; genuine 67.78% |
| Binary accuracy (pick fake from pair) | 85.59% |  |
| Familiarisation effect | +3.84 points (p=0.001) | unary +3.76, binary +3.85 |
| Confidence-weighted AUROC | 73.83% → 75.54% with familiarisation |  |
| Crowd-aggregated AUROC | 95.51% (94.04% with familiarisation) | comparable to in-domain detectors |
| Automated detectors AUROC | in-domain 100%; out-of-domain 25.31% | out-of-domain flagged genuine speech as fake |
| Mandarin vs English listeners | +1.79 points, n.s. (p=0.202) |  |

## Moderators {#moderators}

- Reference audio of the same sentence helped significantly (p<0.001) but is rarely available in practice.
- Extra time and repeated listening did little to improve performance.

## Limitations {#limitations}

- Older TTS (VITS), not current voice cloning
- Short clips
- Participants knew fakes were present

## Cited in {#cited-in}

- [MR-0001](/metareports/human-detection-accuracy.md): Human Detection of Deepfakes: Accuracy, Bias, Confidence, and Training
- [MR-0003](/metareports/voice-cloning-human-detection.md): Voice Clones and Synthetic Speech: How Well Do People Detect Them?

## Verification {#verification}

| Field | Value |
|---|---|
| Status | `verified-fulltext` |
| Compiler confidence | high |
| Checked on | 2026-10-07 |
| Via | journals.plos.org article page |