---
type: source
id: groh-2022-crowds-machines
title: Deepfake detection by human crowds, machines, and machine-informed crowds
authors:
- Matthew Groh
- Ziv Epstein
- Chaz Firestone
- Rosalind Picard
year: 2022
venue: Proceedings of the National Academy of Sciences (PNAS)
volume: 119(1)
pages: e2110013119
doi: 10.1073/pnas.2110013119
arxiv_id: '2105.06496'
peer_review: peer-reviewed-journal
evidence_tier: B
design: Two online studies comparing human crowds with the leading DFDC model; machine-informed condition; face-processing disruption manipulations
modality:
- video
stimuli: Deepfake Detection Challenge (DFDC) videos
n_participants: 15016
n_note: total across two studies
topics:
- human-detection
- human-machine-teaming
- face-processing
summary: Ordinary human crowds and the leading DFDC model were similarly accurate but made different mistakes; model-informed participants beat either alone on average, but inaccurate model predictions often lowered their accuracy.
results:
- metric: Crowd vs leading model
  value: similarly accurate
  note: different kinds of mistakes
  source_location: abstract
- metric: Machine-informed crowd
  value: more accurate than either alone
  note: inaccurate model predictions often decreased accuracy
  source_location: abstract
- metric: Face-processing disruptions
  value: hurt humans, mostly not the model
  note: implicates specialised face processing
  source_location: abstract
moderators: []
limitations:
- Self-selected online samples
- 2019-2020 face-swap deepfakes
verification:
  status: verified-abstract
  confidence: high
  checked: 2026-10-07
  via: arXiv 2105.06496 abstract (journal ref PNAS 119(1))
url: null
pmid: null
pmcid: null
openalex_id: null
oa_url: null
cited_in:
- MR-0001
- MR-0002
updated: '2026-10-07'
---

# Deepfake detection by human crowds, machines, and machine-informed crowds

> **Source record** `groh-2022-crowds-machines` · 2022 · Proceedings of the National Academy of Sciences (PNAS)  
> **Evidence tier:** B (peer-reviewed-journal) · **Verification:** verified-abstract (compiler confidence: high)  
> **Bottom line:** Ordinary human crowds and the leading DFDC model were similarly accurate but made different mistakes; model-informed participants beat either alone on average, but inaccurate model predictions often lowered their accuracy.

## Citation {#citation}

Matthew Groh, Ziv Epstein, Chaz Firestone, Rosalind Picard (2022). Deepfake detection by human crowds, machines, and machine-informed crowds. *Proceedings of the National Academy of Sciences (PNAS)*, 119(1), e2110013119. https://doi.org/10.1073/pnas.2110013119

Verification entry (identifiers, resolvers, claims with source locations): [/data/citations.json#groh-2022-crowds-machines](/data/citations.json#groh-2022-crowds-machines)

## Study design {#design}

| Field | Value |
|---|---|
| Design | Two online studies comparing human crowds with the leading DFDC model; machine-informed condition; face-processing disruption manipulations |
| Modality | video |
| Stimuli | Deepfake Detection Challenge (DFDC) videos |
| Participants (N) | 15016 |
| N detail | total across two studies |
| Topics | human-detection, human-machine-teaming, face-processing |

## Results {#results}

| Metric | Value | Note |
|---|---|---|
| Crowd vs leading model | similarly accurate | different kinds of mistakes |
| Machine-informed crowd | more accurate than either alone | inaccurate model predictions often decreased accuracy |
| Face-processing disruptions | hurt humans, mostly not the model | implicates specialised face processing |

## Limitations {#limitations}

- Self-selected online samples
- 2019-2020 face-swap deepfakes

## Cited in {#cited-in}

- [MR-0001](/metareports/human-detection-accuracy.md): Human Detection of Deepfakes: Accuracy, Bias, Confidence, and Training
- [MR-0002](/metareports/uncanny-valley-and-synthetic-faces.md): The Uncanny Valley and Synthetic Media: Does It Exist, What Causes It, and Have Deepfakes Crossed It?

## Verification {#verification}

| Field | Value |
|---|---|
| Status | `verified-abstract` |
| Compiler confidence | high |
| Checked on | 2026-10-07 |
| Via | arXiv 2105.06496 abstract (journal ref PNAS 119(1)) |

Values on this page have not all been checked against the full text. Confirm them against the primary source before high-stakes use.