This started as a boring chore: merge a OneDrive photo library with a Google Photos export, kill the duplicates, and organize everything by date. It ended as one of the cleaner demonstrations I've run of a principle that has nothing to do with photos — how to transform valuable data from one state to another without ever putting the originals at risk.
The scripts that did the work will age out — a tool changes, an export format shifts, an operating system moves on. But the reasoning behind them is transferable. It applies to code refactors, database migrations, server upgrades, finance reconciliations, mass record updates, and every AI-assisted workflow where the cost of a mistake is high and the cost of being careful is low. So here's the thinking, not the scripts.
The one idea underneath everything
Every design decision came from a single rule: never modify originals until you've proven the new thing is good.
That principle is why there was a staging area to verify the new library before touching the real one; why every destructive step required a typed confirmation; why every deletion went through the Recycle Bin or an archive folder instead of a permanent delete; why every operation wrote a report you could audit afterward; and why each phase finished completely before the next began, so a failure stayed contained instead of cascading. At any point, if something went wrong, there was a path back.
Preview, confirm, execute, audit
The single most important habit in the whole project was that every phase that could change or delete data produced a report first — describing exactly what would happen — before a second pass actually did it. Four beats, every time:
That rhythm is not a photo pattern. It's a financial transaction (show the debit, confirm, execute, receipt). It's a database migration (dry-run, review, apply, migration log). It's a customer email send, a code deployment, a bulk personnel change in an HR system. Any bulk operation that touches something valuable should move through those same four beats. The preview step alone catches most disasters, because it turns "I think this is right" into "I can see what this will do."
It nearly earned its keep on day one. When the keep-a-file rule first ran, the previewed samples looked alarming — the files marked for deletion had tidy, organized names while the keepers had odd ones. It looked broken. Ten more sample cases revealed the rule was actually correct: it was preserving the curated originals and discarding auto-imported copies, just on a counterintuitive signal. Without the preview, that rule would have run blind. With it, the scare cost five minutes instead of a library.
Always preview a destructive operation against real data before running it — and read enough samples to see the pattern, not just the first row. The first row will fool you.
When you need certainty, hash — don't guess
The dedup engine compared files by cryptographic hash, not by name, size, or date. That mattered because all three of the easy signals lie. Filenames lie — "IMG_1234 (2).jpg" might be identical to the original or completely different. Sizes lie — two different photos can happen to be the same byte count. Dates lie — copying a file resets its modified date; editing rewrites its capture date. A hash compares the actual bytes and answers definitively: identical, or not.
When you need to be sure two things are the same, compare their contents, not their labels. Metadata is a hint; the bytes are the truth.
Archive beats delete when storage is cheap
The plan started with "delete the old folders." Mid-run it shifted to "move them into a dated archive and keep them as cold storage in the cloud." That turned out to be the better default whenever storage isn't the binding constraint. It's the same soft-delete pattern that databases, email trash folders, and the Recycle Bin already use — just applied at the bulk-folder level, with the cloud as the soft-delete tier. You get bulletproof undo, you make no permanent decisions under time pressure, and you can revisit the archive months later once you're genuinely confident.
The cost asymmetry is the whole argument: storage is cheap, and recovering from a deletion you regret is expensive. When one side of a trade-off is that lopsided, default to the cautious side. (Delete still wins when space is tight, when retention rules force it, or when keeping copies of sensitive data is itself the liability.)
Respect the guardrails — even the accidental ones
Late in the project, a bulk move in the cloud web interface was rejected with a "file count over limit" error. Annoying — until a closer look showed the selection had accidentally included the master library itself. The move would have swept the finished work in with the folders being retired. A limit built for server load, with no safety intent at all, prevented a real disaster.
Systems are full of these accidental safety nets: rate limits that stop a runaway script, permission errors that block writes to system files, disk quotas that force a pause before a volume fills. The instinct when a guardrail is "in the way" is to disable it — turn off the prompt, raise the limit, suppress the warning. Resist that. When a system tells you an operation is too big or too risky, the right response is to slow down and verify, not to find a workaround.
State is a moving target — build for it
A file-count check after sync showed roughly 1,700 files apparently missing, and triggered a real investigation. The truth: nothing was missing. The count had been taken before cloud sync finished bringing files back down in the background. By the time the recovery script ran, it correctly reported "0 copied, 1,410 skipped — already in place." The most boring possible output, which is exactly what you want from a recovery step.
Two lessons compounded there. First, in any distributed or cloud-synced system, the state you observe is state at a point in time, not state "now" — so re-verify immediately before you act, never on a count from an hour ago. Second, that's exactly why every write should be idempotent: a "skip if it already exists" check on every operation meant a script that was safe to run twice, and safe to run against a system that had quietly fixed itself. The same discipline governs APIs with eventual consistency, replicated databases, and message queues. Assume the ground can move under you, and build so that it's fine when it does.
The transferable checklist
Strip away the photos and the specific tools, and the whole approach reduces to twelve habits that work for code refactors, database migrations, server upgrades, finance reconciliations, mass record updates — anything where you're transforming valuable data without breaking it.
- Inventory before acting. Understand the data before you modify it.
- Hash, don't guess. Use deterministic comparison when accuracy matters.
- Use existing tools when they exist. Don't rebuild what's been battle-tested.
- Preview before destructive operations. Every time. No exceptions.
- Soft-delete by default. Recycle Bin, archive folder, or a retention window — give yourself an undo.
- Generate audit trails. Every operation produces a report you can look back on.
- Confirm before commit. Require a deliberate action — typing "DELETE," not a reflex click.
- Stage in safe space, then move once. Don't work in production.
- Verify before cleanup. The new thing must be proven before the old thing dies.
- Build idempotent steps. Safe to run multiple times; skip-if-done at every write.
- Respect distributed state. Re-verify before acting; the system changes while you work.
- Plan for recovery. Build the recovery path during design, not during the crisis.
None of the major mistakes happened. The library came out consolidated and triply backed up, the originals archived safely in the cloud. But the win wasn't the clean library — it was the confirmation that these habits hold at any scale. Preview-confirm-execute-audit, hash for certainty, archive over delete, respect the guardrails, assume state moves. They're general-purpose problem-solving habits for high-stakes, low-drama work.
The scripts will age out. The patterns won't.