I built a YouTube upload stage for a video pipeline, and the flow looked clean enough on paper: start a resumable session, PUT the file, get back a video ID, verify the upload actually landed the way it was supposed to, then write a local record marking the episode as uploaded. Four steps, each one depending on the last. It was the dependency between the last two that turned out to be the problem.
Verification here means re-querying the video through videos.list after the upload finishes, to confirm the visibility wasn't silently demoted, the upload wasn't rejected, and the metadata actually propagated. That's a reasonable thing to check — YouTube's upload API can report success at the transport layer while the platform-side processing does something you didn't ask for. But if that verification call raises, the exception propagates straight up, and the local record — a JSON file I'll call upload.json — never gets written. Not "gets written with an error flag." Never written, period.
By the time that exception fires, though, the video already exists on YouTube. The PUT succeeded. The video ID is real. There's a public (or not-quite-public) video sitting on the channel, and there is exactly nothing on disk that knows about it.
Run the same command again after that, and the guard that's supposed to answer "have I already uploaded this?" — a check for whether upload.json exists — sails right through, because it doesn't exist. The result isn't a retry. It's a second, completely independent upload of the same video.
The module's docstring said retries don't create duplicate videos. That line wasn't wrong, exactly — it was scoped narrower than it read. It was true for retries inside the low-level file-PUT function, which reuses the same resumable session URI on retry, so transport-layer hiccups during the upload itself are genuinely safe to retry. What it didn't cover was someone re-running the whole command from the outside after a failure. That's a completely different code path, going through the duplicate guard instead of the resumable session, and nothing about the inner retry safety extended to it. One sentence in a docstring, two different meanings of "retry," and only one of them was actually protected.
While tracking this down, I found a second instance of the identical root cause already sitting in the same repository. Before this upload stage existed, five episodes had already gone up through a different path — an MCP tool — and that path never wrote upload.json either. Which meant the brand-new duplicate guard wasn't just vulnerable to a future verification failure; it was already blind to five specific videos that were live on the channel right now. A single run of the new command, pointed at any of those five, would have found no record, seen no reason to stop, and put a duplicate of each one on the channel. The defect wasn't a bug in the guard's logic. The guard's logic was fine. The defect was that a second path could create the same kind of resource and never tell the first path's bookkeeping about it.
It's worth pausing on how this got past testing at all, because every relevant unit test was green the whole time. Happy path, the missing-file guard, the public-visibility check, the duplicate guard, even a test asserting that verification throws when it detects a demotion — all covered, all passing. What none of them checked was what state got left on disk after that exception fired. Tests tend to ask what a function returns and what it throws. Whether the right thing got thrown is easy to assert. What's still sitting on the filesystem once it has been thrown is a different question, and it's one that residual-state bugs like this one live in specifically because nobody's assertion is pointed at it.
The fix moved the checkpoint, not the check. As soon as the video ID comes back — before verification runs at all — the code now writes upload.json immediately, with a field marking it verified: false. Verification then runs against that already-recorded video ID. If verification succeeds, the record gets updated to verified: true. If it fails or the process dies, there's already a record on disk saying "this video exists, id X, not yet confirmed."
That record turns the duplicate guard into three branches instead of one: --force means deliberately create another copy, a record with verified: false means resume verification against the existing video ID rather than uploading again, and anything else means block the run outright because the episode is already up. Re-running the command after a verification failure no longer re-uploads anything — it just finishes the check it didn't get to finish the first time.
The five already-live episodes needed a different move, since there was no failed-verification moment to catch retroactively — they'd gone up through a path that never wrote a record at all. For those, I backfilled upload.json entries using the real API values for each video, so the guard had something to check against going forward. Rearming a guard after the fact only works because the actual state (the video, its ID, its visibility) was still recoverable from the platform. If it hadn't been, there'd have been no way to reconstruct which videos were already live without visually checking the channel.
Any operation that creates a resource somewhere remote and then verifies it has a window — the gap between the moment the side effect is committed out there and the moment that fact gets recorded in here. Die inside that window, and two states become indistinguishable from the outside:
Retry logic is built entirely on the assumption that you can tell these apart. When you can't, "retry" and "duplicate" become the same button. This isn't specific to video uploads — payment authorization, cloud instance provisioning, creating an account on some third-party service, any file upload with a follow-up confirmation step: different resource, same shape of window, same failure mode if nothing gets written until after the confirmation.
The rule that falls out of it is simple to state and easy to skip in practice: the moment you receive an identifier is the moment you record it, before verification, before any post-processing, before cleanup. The record shouldn't only be capable of saying "done" — it needs a field for "this happened, not yet confirmed," because that's the actual state the system is in for however long the window stays open. And the diagnostic question is just as simple: if this function dies on any given line, does re-running it produce the same result? Ask it specifically for every line between the one where the remote write succeeded and the one where that fact gets written locally. The longer that stretch of code, the more likely something in it eventually dies mid-way.
One more piece, easy to miss because it isn't about the retry path at all: if two different code paths can create the same kind of resource, both of them owe the same bookkeeping. A guard built with only one creation path in mind will only ever know what that one path told it — which is exactly how five already-uploaded videos ended up invisible to a duplicate check that, on its own terms, was working correctly.
A related note on this site covers the same pipeline shipping a different kind of invisible bug: a width guard that measured text the renderer never actually drew, with every test staying green the whole time. Different mechanism, same shape — the tests weren't wrong about what they checked, they just weren't checking the thing that broke.
Q. Why did a failed verification step lead to a duplicate video upload?
Because the local record marking an episode as uploaded was only written after verification succeeded. When verification raised an exception, the record was never written, even though the video already existed on YouTube. Re-running the same command found no record, so the duplicate guard passed and a second, independent copy was uploaded.
Q. What was the actual fix?
The checkpoint moved earlier: as soon as the video ID comes back, before verification runs, the record is written immediately with verified: false. Verification then updates it to verified: true on success. Re-running the command after a failure resumes verification against the existing video ID instead of uploading again.
Q. Why didn't the existing tests catch this?
Every unit test was green — the happy path, each guard, and even the assertion that verification throws when it detects a demotion. None of them checked what state was left on disk after that exception fired. Tests checked what was returned and thrown, not what a failure path left behind on the filesystem.
Q. Does this generalize beyond video uploads?
Yes. Any operation that creates a resource remotely and then verifies it has a window between the side effect being committed and that fact being recorded locally. Payment authorization, cloud instance provisioning, external account creation — different resource, same window, same failure mode if nothing gets recorded until after verification.