anky8998

One thing we ran into while working on similar problems is that static opcode scanning alone tends to give a false sense of coverage. A lot of real-world bypasses don’t rely on obvious GLOBAL os.system patterns and can evade tools that depend on pickletools, modelscan, or fickling.

We recently open-sourced a structure-aware pickle fuzzer at Cisco that’s designed specifically to test the robustness of pickle scanners, not just scan models:

• It executes pickle bytecode inside a custom VM, tracking opcode execution, stack state, and memo behavior • Mutates opcode sequences, stack interactions, and protocol-specific edge cases • Has already uncovered multiple scanner bypasses that look benign statically but behave differently at runtime

Why We Built It

Pickle remains widely used to distribute ML models (PyTorch, sklearn, etc.), and scanners are supposed to catch unsafe reducers before loading. We wanted to know: how effective are these scanners against adversarially modified pickle files?

What We Found

During fuzzing, we discovered multiple zero-day bypasses in tools like:

ModelScan

Fickling

Other pickle-security scanners

All issues were responsibly disclosed and patched.

How the Fuzzer Works

It’s “structure-aware,” meaning it can:

mutate opcodes without invalidating the pickle

preserve stack state & memo indices

generate nested / self-referential graphs

create realistic malicious reducer chains

This allows it to surface detection gaps that signature-based or AST-reconstruction tools often miss.

Why It Matters

Model files are now part of the AI supply chain. A single malicious pickle can lead to RCE or environment compromise. Hardening scanners requires adversarial testing — not just validation that a file parses.

Links