"Duplicates" covers two situations that need different handling.

Exact Duplicates

Byte-identical files, usually from:

  • Importing the same memory card twice.
  • Restoring a backup on top of an existing library.
  • Copying a folder rather than moving it.
  • Cloud sync conflicts producing "photo (1).jpg".

These are safe to remove automatically, because the remaining copy is genuinely identical. Detection works by comparing file hashes — a fingerprint of the file's contents — so matches are certain rather than probabilistic.

Near-Duplicates

Visually similar but not identical files:

  • Burst sequences and rapid-fire shots.
  • The same photo at different sizes — an original plus a compressed version you shared.
  • The same photo edited differently.
  • A photo and its screenshot.
  • The same image re-saved by a chat app.

These need human review, because the "duplicates" often differ in exactly the way that matters. Detection uses perceptual hashing, which measures visual similarity — inherently a judgement about a threshold, not a certainty.

The Trap That Loses Originals

The dangerous case is the same image at different qualities. A tool sorting by "keep the newest" or "keep the one in this folder" may retain the 800 px compressed copy you posted online and delete the 24-megapixel original.

The rule: when two files look identical, keep the largest one — highest resolution and largest file size. Configure your deduplication tool to prefer that, and check the setting before running anything in bulk.

A Safe Deduplication Process

  1. Back up first. Non-negotiable. Deduplication is bulk deletion by another name.
  2. Handle exact duplicates first, automatically, by hash. This is the safe majority.
  3. Then review near-duplicates in batches, with a preview large enough to actually judge — sharpness and expression differences are invisible in a small thumbnail.
  4. Prefer "move to a holding folder" over deleting. Keep it for a few weeks before emptying.
  5. Check the "keep" criteria: highest resolution, original format, earliest date, and — importantly — the one your catalogue has edits or ratings attached to.
  6. Do it before organizing, not after, so you don't file the same photo repeatedly.

Preventing Accumulation

  • Import from cards with "skip duplicates" enabled, which most photo managers support.
  • Move rather than copy when reorganizing.
  • Cull bursts at import. Choosing one frame from thirty is fastest while you still remember the moment.
  • Keep one library, not several partial ones on different drives — split libraries are the main source of large-scale duplication.
  • Don't save images you shared. The compressed copy you sent has no value if you have the original.

Do You Even Need To?

Storage is cheap; your attention isn't. If duplicates aren't causing a real problem — full drive, cluttered search results, confusion about which version is current — the effort may not be worth it. Deduplicate when there's a reason, back up before you do, and prefer keeping too much over deleting something irreplaceable.