---
type: source
id: groh-2024-political-speech
title: Human detection of political speech deepfakes across transcripts, audio, and video
authors:
- Matthew Groh
- Aruna Sankaranarayanan
- Nikhil Singh
- Dong Young Kim
- Andrew Lippman
- Rosalind Picard
year: 2024
venue: Nature Communications
volume: '15'
pages: '7629'
doi: 10.1038/s41467-024-51998-z
peer_review: peer-reviewed-journal
evidence_tier: B
design: 5 pre-registered experiments; real vs fabricated political speeches shown as transcript, silent video (± subtitles), audio (± subtitles), or video+audio (± subtitles)
modality:
- text
- audio
- video
- audiovisual
stimuli: Real and fabricated political speeches; fabricated audio from state-of-the-art TTS or voice actors
n_participants: 2215
n_note: 2,215 recruited participants across 5 experiments, plus 41,313 non-recruited participants in Exp 1b (Exp 1a recruited n=501)
topics:
- human-detection
- modality
- voice
summary: Accuracy rose with each added modality (transcript 57% → silent video 64% → audio 81% → video+audio 85-86% in Exp 1a); people relied more on how something was said than on what was said, and TTS fakes were harder to spot than voice-actor fakes.
results:
- metric: 'Exp 1a (recruited, n=501): transcript'
  value: 57%
  note: content alone ≈ slightly above chance
  source_location: null
- metric: 'Exp 1a: silent video'
  value: 64%
  note: ''
  source_location: null
- metric: 'Exp 1a: audio'
  value: 81%
  note: with or without subtitles; Discussion reports 80.5%
  source_location: null
- metric: 'Exp 1a: video + audio'
  value: 85% (with subtitles), 86% (without)
  note: ''
  source_location: null
- metric: 'Exp 1b (non-recruited, n=41,313): transcript / silent video / audio / video+audio'
  value: 43% / 65% / 74% / 82-83%
  note: ''
  source_location: null
- metric: 'Exp 3: transcript / subtitled video / audio / video+audio'
  value: 58% / 62% / 65% / 74%
  note: same ordering at lower levels
  source_location: null
- metric: 'Exp 2: TTS vs voice-actor deepfakes identified'
  value: 72% vs 83%
  note: state-of-the-art TTS harder to detect
  source_location: null
- metric: Cue reliance
  value: delivery (audio-visual) > content
  note: authors' conclusion
  source_location: null
moderators:
- Participants struggled to distinguish a speaker's real audio from TTS trained on that speaker.
limitations:
- Political figures only (familiar speakers may aid detection)
- Participants knew fakes were present
verification:
  status: verified-fulltext
  confidence: high
  checked: 2026-10-07
  via: nature.com article page (open access); author list on page truncated, middle authors from compiler knowledge
url: null
pmid: null
pmcid: null
arxiv_id: null
openalex_id: null
oa_url: null
cited_in:
- MR-0001
- MR-0003
updated: '2026-10-07'
---

# Human detection of political speech deepfakes across transcripts, audio, and video

> **Source record** `groh-2024-political-speech` · 2024 · Nature Communications  
> **Evidence tier:** B (peer-reviewed-journal) · **Verification:** verified-fulltext  
> **Bottom line:** Accuracy rose with each added modality (transcript 57% → silent video 64% → audio 81% → video+audio 85-86% in Exp 1a); people relied more on how something was said than on what was said, and TTS fakes were harder to spot than voice-actor fakes.

## Citation {#citation}

Matthew Groh, Aruna Sankaranarayanan, Nikhil Singh, Dong Young Kim, Andrew Lippman, Rosalind Picard (2024). Human detection of political speech deepfakes across transcripts, audio, and video. *Nature Communications*, 15, 7629. https://doi.org/10.1038/s41467-024-51998-z

Verification entry (identifiers, resolvers, claims with source locations): [/data/citations.json#groh-2024-political-speech](/data/citations.json#groh-2024-political-speech)

## Study design {#design}

| Field | Value |
|---|---|
| Design | 5 pre-registered experiments; real vs fabricated political speeches shown as transcript, silent video (± subtitles), audio (± subtitles), or video+audio (± subtitles) |
| Modality | text, audio, video, audiovisual |
| Stimuli | Real and fabricated political speeches; fabricated audio from state-of-the-art TTS or voice actors |
| Participants (N) | 2215 |
| N detail | 2,215 recruited participants across 5 experiments, plus 41,313 non-recruited participants in Exp 1b (Exp 1a recruited n=501) |
| Topics | human-detection, modality, voice |

## Results {#results}

| Metric | Value | Note |
|---|---|---|
| Exp 1a (recruited, n=501): transcript | 57% | content alone ≈ slightly above chance |
| Exp 1a: silent video | 64% |  |
| Exp 1a: audio | 81% | with or without subtitles; Discussion reports 80.5% |
| Exp 1a: video + audio | 85% (with subtitles), 86% (without) |  |
| Exp 1b (non-recruited, n=41,313): transcript / silent video / audio / video+audio | 43% / 65% / 74% / 82-83% |  |
| Exp 3: transcript / subtitled video / audio / video+audio | 58% / 62% / 65% / 74% | same ordering at lower levels |
| Exp 2: TTS vs voice-actor deepfakes identified | 72% vs 83% | state-of-the-art TTS harder to detect |
| Cue reliance | delivery (audio-visual) > content | authors' conclusion |

## Moderators {#moderators}

- Participants struggled to distinguish a speaker's real audio from TTS trained on that speaker.

## Limitations {#limitations}

- Political figures only (familiar speakers may aid detection)
- Participants knew fakes were present

## Cited in {#cited-in}

- [MR-0001](/metareports/human-detection-accuracy.md): Human Detection of Deepfakes: Accuracy, Bias, Confidence, and Training
- [MR-0003](/metareports/voice-cloning-human-detection.md): Voice Clones and Synthetic Speech: How Well Do People Detect Them?

## Verification {#verification}

| Field | Value |
|---|---|
| Status | `verified-fulltext` |
| Compiler confidence | high |
| Checked on | 2026-10-07 |
| Via | nature.com article page (open access); author list on page truncated, middle authors from compiler knowledge |