Bit 0 is not a coin

The assumption

An encrypted payload is indistinguishable from random bits. Write it into the lowest bits of a model's weights and each of those bits becomes a coin flip: set half the time. The check called S4 rests on that. For each tensor it measures how often each low bit is set, and asks whether that rate has been pulled to 50%.

For that to mean anything, the clean bits must not already sit at 50%. In ResNet-18 they do, which is why no single-file check can see a payload there. The first S4 went one step further and assumed every model is like that: that 50% is what clean weights look like, and a large departure from it is the thing to report.

What the model said

The first false-positive run used two unmodified torchvision models. ResNet-18 raised 0 alarms. MobileNetV3-Small raised 150, out of 210 float tensors.

Nothing was hidden in that file. Its lowest bit is set in ~67% of all weights, and the next bit up in ~37%. Every tensor carries the same lean. We do not know where it comes from, and the check does not need to: what matters is that it is consistent across the model, so a tensor that loses it stands out.

The mistake was not the threshold. It was the reference. A clean model is not a fair coin. It is whatever its other tensors are.

First fix: the model is its own reference

S4 was rewritten to compare each tensor with the median of the model's own tensors, allowing for the spread between clean ones. That rule was chosen after seeing MobileNetV3, so it was then run on 7 torchvision models nobody had looked at. 1 of them raised a false alarm (ShuffleNetV2's classifier bias). It was reported, not tuned away.

Then the survey reached half-precision files on Hugging Face, and the alarms came back. Different kinds of tensor carry different signatures (embeddings and routers sit apart from the rest), so S4 now reads each tensor against its role peers: the same weight in the other layers. That rule was frozen and run once on a fresh batch of models committed before their weights were read. For bf16 files, models with a false alarm went from 14 → 4 of 20. For fp16, 5 → 5 of 10. The fix did nothing at all for fp16.

Which way, not how far

The fp16 alarms that survived had one thing in common. In almost every case the flagged bit was further from 50% than its peers: a bit stuck at zero in a quantized embedding, a norm layer with fewer than half its low bits set.

A random payload cannot do that. It only adds entropy, so it can only pull a bit toward 50%, never away. Every one of those alarms was moving in a direction no payload can move. S4 had been a two-sided test for a one-sided effect.

So S4 now flags a bit only when it is nearer 50% than its reference. That rule was frozen and run once on a third fresh batch: bf16 models with a false alarm went 5 → 1 of 20, and fp16 1 → 0 of 20. Each batch is used once. Any further change to S4 needs a fourth.

What the fix cost

A one-sided test gives something up. A payload of zeros, a constant or plain text pushes bits away from 50%, exactly like low precision does, and from one file the two cannot be told apart. S4 no longer sees those. The diff against the claimed base model (S5) still does, because it does not care which way a bit moved, only that it moved where training does not.

It also leaves the original problem standing. In 23 of the 39 popular float32 models surveyed, the low bits really are fair coins, and there a payload moves nothing. S4 stopped crying wolf. That is not the same as seeing wolves, and on most models it cannot.