← Writing

The 137,138-byte PNG that was actually JSON

A production PDF bug that traced back through a misleading success log, a corrupted logo file, and a null-coalescing operator — and why I refuse to trust a fix until it has been run against every row of production data.

On this page

A clinic reported that their visit-note PDFs had stopped generating. The error, buried in logs, was oddly specific:

Unable to guess the image type 137138 bytes

The PDF library was trying to embed the clinic’s logo and couldn’t figure out what kind of image it was looking at. Which was strange, because the file was called clinicLogo.png.

This is the story of how that one error message unwound into a data-integrity bug affecting 36 production clinics, and the two lessons it hammered home: null and empty string are different bugs, and a fix you haven’t verified against all of production is a hypothesis, not a fix.

The misleading success log

The first thing the logs said was that everything was fine. The generation path logged “PDF Was Generated” — a success message — right before the failure surfaced downstream. That message turned out to mean something much weaker than it claimed: a stage of the pipeline had completed, not that a valid document existed. The first hour of debugging was spent trusting a log line that was lying by omission.

Rule of thumb since: a success log should assert the outcome (“wrote 84,213-byte PDF with embedded logo”), not the fact that a function returned.

The byte count as a fingerprint

The error had one gift in it: 137,138 bytes. That’s an unusual size for a clinic logo — most were tens of kilobytes. Whatever the PDF library was choking on, it was a specific file with a specific size, which meant it could be found and inspected.

Pulling the cached file out of the app’s storage on the simulator and opening it in a text editor answered the question instantly. clinicLogo.png wasn’t a PNG. It was JSON — specifically, a Firebase Storage directory-listing response that had been saved to disk under an image filename. At some point the download code had requested a listing instead of an object, got a perfectly valid HTTP 200 with a JSON body, and faithfully wrote those bytes to clinicLogo.png. Every downstream consumer trusted the extension.

The image library was right all along: 137,138 bytes of JSON is not any image type it should be able to guess.

Upstream: the fallback that could never fire

Why was the app fetching a directory listing in the first place? Because the clinic’s stored logo path was an empty string.

The settings screen read the logo path through a state getter that did this:

valueOrNull ?? ""

That ?? "" looked like defensive programming. In practice it made the real safety check downstream — “if the path is null, keep the existing stored value” — into dead code. The getter never returned null, so the null-guard never triggered, so every time an affected clinic saved their settings, the app wrote "" over their perfectly good cloudStoragePath. Silently. No error, no log, no visible change until the next PDF tried to embed a logo from an empty path and got a directory listing instead.

Null meant “not loaded yet — don’t touch the stored value.” Empty string meant “the user cleared this field — overwrite it.” Collapsing the two with a coalescing operator merged two different states into the destructive one.

Verifying against all of production, not in theory

The fix had three parts: remove the coalescing so null stays null, add a shared magic-byte validator that inspects the first bytes of a file instead of trusting its extension, and write a backfill script to restore the wiped paths.

But before shipping any of it, two questions needed empirical answers.

How many clinics were actually affected? Not “probably a few” — a number. A script classified all 396 production clinics: 36 had been wiped to empty string by the bug, 190 were null because they’d simply never uploaded a logo, and 169 had valid paths. That 190-vs-36 split mattered enormously — a naive “fix every falsy path” backfill would have invented data for 190 clinics that never had any.

Would the new validator reject anything it shouldn’t? My worry was the opposite failure: shipping a validator that starts denying correct images. So the exact magic-byte checks were run against the first 16 bytes of all 199 logo objects in the production bucket. 198 passed. The single rejection was a genuine HEIC file — which was already breaking PDF generation for that clinic today. The validator’s one “failure” was a true positive that surfaced a second, unreported instance of the same class of bug.

Only after both of those runs did the fix ship.

What I took from it

Null and empty string are different bugs. Any time you see ?? "" or ?? 0 near persisted state, ask which downstream check it just turned into dead code. A “safe default” that erases the distinction between absent and deliberately empty is how data gets silently destroyed on every save.

Extensions are labels, not facts. Files are what their bytes say they are. A magic-byte check is a few lines and would have turned a confusing PDF-generation error into a precise “logo file is not an image” error at upload time, months earlier.

A byte count can be a fingerprint. The most useful thing in the whole incident was the number 137,138. Specificity in error messages is what makes them traceable — preserve it.

Don’t trust the fix — prove it against production. Classifying all 396 clinics and validating all 199 images took maybe an hour of scripting. It converted “this should fix it” into “36 clinics restored, 190 correctly untouched, one pre-existing bad file found as a bonus.” That hour is the difference between closing an incident and merely pausing it.

← All writing Book a call →
Book a call → WhatsApp