What can hide in a model's weights that no scanner sees?
Stowaway inspects .safetensors, .pt and .gguf files for payloads in the weights, unaccounted bytes and unsafe pickle code — entirely in your browser, nothing uploaded — and shows that reading one file is not enough.
of random data in the low bits of ResNet-18's weights leaves top-1 accuracy exactly unchanged (66.8% before and after). Tested for detection from 1 KB to 1 MB, the single-file checks missed every size; only a diff against the base model caught them.
The problem
Model scanners such as picklescan and ModelScan look for code hidden in pickle data, and converting a model to .safetensors removes code entirely — so safetensors files are trusted by default.
But a payload does not need to be code. It can sit in the low bits of the weights, in bytes appended after the last tensor, or in alignment padding. It survives format conversion because it is just numbers, and a loader that later pulls it out and runs it is small enough to live anywhere. Stowaway looks where the code scanners do not, and measures how much can hide before anything notices.
What it checks
Pickle imports
Unsafe code in pickle data, found by a restricted unpickler and an opcode walk. Nothing is ever executed.
Byte coverage
Every byte must belong to a header, a tensor or zero padding. Anything else is reported with its offset.
Precision history
Low bits that should be zero — e.g. fp16 or bf16 weights saved as float32 — but are not.
Bit-plane statistics
Low bits pulled toward a fair coin, measured against the tensor's role peers in the same model.
Base-model diff
A bit-level diff against the claimed base model: changes confined to the low bits are a payload, not training.
Formats are detected from content, never the extension: safetensors, PyTorch zip (.pt/.pth/.bin), plain pickle and GGUF.
The experiments
A payload is just numbers. It survives conversion to a "safe" format, costs the model nothing measurable, and no code scanner looks at it. So the question is which models give it away from the file alone.
| What the weights look like | Models | Payload seen from the file alone |
|---|---|---|
| float32 file carrying only fp16 or bf16 precision | 14 | 14 / 14 at every k |
| float32 with a low-bit signature | 2 | 2 / 2 at every k |
| float32 with fair-coin low bits | 23 | 0 / 23 at every k |
The 40 most-downloaded non-gated float32 safetensors models on Hugging Face (39 had testable samples); random bytes in the low k = 1, 4 or 8 bits of one sampled tensor. For 23 of 39, a clean single-file scan certifies nothing about the weights.
What one file can't see
Random bytes in ResNet-18's low 8 bits, up to 4 MB, leave top-1 accuracy exactly unchanged, and from 1 KB to 1 MB they pass both single-file checks at every size. Only a bit-for-bit diff against the base model catches them.
Across all 9 torchvision models and five payload types (random, zeros, text, compressed, an executable-like blob), a payload of up to 1 MB in the low 8 bits left at least 99.8% of top-1 predictions unchanged, and the base diff caught 648 / 648 planted copies of random, zero and text payloads.
MobileNetV3-Small's low bits carry a signature (bit 0 set in ~67% of values) that a random payload disturbs, so the naive attack is seen from the file. An attacker who knows the checks can still evade them: spreading the payload over one weight in 64 keeps every statistic inside the threshold at every size that fits at that density, up to 16 KB. The base diff catches that from 4 KB.
Four hypotheses
Hypotheses were written before any experiment ran (PLAN.md).
What we learned
A clean scan is not a clean model.
For most popular float32 models, single-file statistics have no signal: a payload thin enough moves no number far enough. The check that does not depend on where the bytes go needs the trusted base model to diff against.
Whether one file is enough depends on the model.
Some models' low bits are fair coins that hide anything; others carry a signature a payload disturbs. We measure which, rather than assume.
A third of "float32" uploads are really half precision.
14 of 39 top models are saved as float32 but hold only fp16 or bf16 precision, leaving a block of zero low bits — exactly where a payload would go, and exactly where S3 sees it.
The base diff passed on samples and failed on whole files.
Calibrated on sampled tensors, S5 had no false alarms. On whole files it flagged 8 of 12 honest fine-tunes: tensors that barely train, such as attention key biases, change only in their low bits. The fix asks how a payload changes bits, not how far: about half its changes stop at one bit. Chosen on those 12 pairs and run once on 20 fresh ones, it flags none and misses no payload the old rule caught in an untouched tensor; inside trained tensors it still misses most small-k payloads.
Validated on unseen models
scanned as whole files, from four bases never used before (MiniLM, ALBERT, DistilRoBERTa, BERT-cased). The earlier S5 rule flagged 16 of them.
fresh pairs where S5 caught a payload in a tensor identical to the base, at every k tested (8, 16, 20 and 21).
on a fresh batch of bf16 models, first revision of S4 against the current rule (and 1 → 0 of 20 for fp16).
Each detector was frozen, then run once on a batch of models committed before their weights were read, so these numbers were not tuned on the models they describe.
Reproduce
Nothing is trained: models are downloaded pretrained and only run forward. Every run appends to results/ledger.jsonl with the git commit that produced it, and docs/RESULTS.md is generated from that ledger, never edited by hand. 237 tests: 126 Python, and 111 for the browser build, including parity tests that require byte-identical JSON reports to the Python tool on every fixture.
Limitations
- /Two vision models end to end, one seed, and random payloads only; the E5 scanner comparison uses a small corpus we built, so it shows what each tool can see, not rates in the wild.
- /Single-file statistics cannot see a small or thinly spread encrypted payload in a fair-coin model.
- /The base diff needs the real base model, and misses a payload inside a tensor the fine-tune genuinely trained.
- /The survey tests 64 KB samples, not whole files, and is weighted to the most downloaded models, not a random sample.
- /Both scanners hold one tensor in memory at a time (and its base counterpart for S5), so files of any size scan, but a single tensor larger than the memory available cannot be checked.