Media Dedupe

Media Dedupe

Media Dedupe keeps your media library clean by preventing duplicate media entities from ever being created. When an
editor uploads a file whose contents are byte-for-byte identical to one already stored, Media Dedupe transparently
references the existing media entity instead of creating a new duplicate - no error message, no manual decision, no
clutter.

Detection is content-based (a cryptographic hash of the file), so it works regardless of the file name, and it covers
both the Media Library widget and the standalone /media/add/* forms.

Key features

- Content-hash matching (SHA-256 via File Hash https://www.drupal.org/project/filehash) - not filename-based, so
renamed-but-identical files are still caught and different files that happen to share a name are not falsely flagged.
- Transparent auto-reference - the existing media item is reused automatically; the editor is simply pointed at it. No
"file already exists" dead-end.
- Covers both upload paths - the Media Library upload dialog and the standalone media add form (/media/add/{type}).
- Per-media-type control - enable deduplication globally or restrict it to specific file-based media types.
- Drupal 10 & 11, clean dependency-injected code, passes phpcs (Drupal/DrupalPractice) and PHPStan, and ships with a
Kernel test.

How it works

File hashes are computed and stored by the File Hash module. On upload, Media Dedupe looks up the uploaded file's hash
among existing permanent files. If a match is found, the Media Library and media-add forms are overridden so that the
existing media entity is referenced (or the user is redirected to it) instead of a new duplicate being persisted. If
no match is found, the file is stored normally.

Backfill hashes for pre-existing files once with drush filehash:generate.

Requirements

- Drupal ^10 || ^11
- File Hash (https://www.drupal.org/project/filehash) with at least one algorithm enabled (SHA-256 recommended)
- Core Media Library

Configuration

Settings live at Administration → Configuration → Media → Media Dedupe (/admin/config/media/media-dedupe): a master
on/off switch and an optional restriction to specific media types.

How this differs from similar modules

Several modules address media duplicates, but with a different strategy:

- Media Duplicate Check and similar detect by filename and warn the editor.
- Media Dupe Detect hashes files but only suggests the existing item.

Media Dedupe is the only one that combines byte-exact content hashing with automatic referencing of the existing
entity across both upload paths - the duplicate simply never gets created.

Limitations

- Matching is byte-exact: visually similar but re-encoded/resized images are treated as different files.
- Deduplication is scoped per media type.
- Only local, file-based sources are supported (remote/oEmbed sources such as YouTube are out of scope).