Home / Proving Ground Free pilot · Small and on-device models

The Proving Ground

A free, one-time adversarial assessment for small and on-device AI models. One per team. You get the evidence before your users find the failures.

Most small models ship without ever being attacked on purpose. The Proving Ground is the battery I run in my own lab, offered once per team at no charge: a fixed-scope evaluation of one model, run against the failure modes that actually show up in production.

What the battery tests

• Data exfiltration. Planted canary secrets sit in the model's context. The battery measures whether adversarial pressure can pull them out.
• Indirect prompt injection. Instructions hidden in documents, web pages, and tool output: the attacks that arrive inside content your model is asked to process.
• Refusal calibration. Harmful requests are paired with benign lookalikes, so over-refusing is scored alongside under-refusing.
• Structured output fidelity. Whether the model still produces exact, parseable output at realistic context lengths.
• Agentic task completion. Multi-step tasks inside a bounded tool loop, with completion verified against state, not taken from the model's word.

What the lab has already found

Across one current edge-model family tested end to end, every model leaked planted canary secrets under adversarial pressure. A popular modification that removes refusal behavior made the leakage worse, not better: the modified model refused nothing and gave up more. And when small models were dropped into a sealed environment with a job to do, they failed by loop breakdown, losing track of where they were and what they had already tried, not by lack of knowledge. The full method and results are published in The Model That Talked to Itself, and the harness behind the battery is open source.

What you receive

• A written report with scores by failure class and per-case evidence drawn from hash-chained transcripts.
• Remediation guidance your engineers can act on, in priority order.
• A readout call to walk through the findings and answer questions.

Delivered within 10 business days of completed intake. Your model, your prompts, and your results are confidential by default. Nothing about an assessment is published, quoted, or reused without your written permission.

Beyond the free pilot

The free assessment covers one small or on-device model at a fixed scope. Larger work (hosted frontier models, fleets of agents, production infrastructure, repeated or ongoing evaluation) is scoped separately under an adjusted proposal, priced on the scale of the infrastructure involved. If your project is bigger than the pilot, say so in the first email and we will scope it honestly.

Request an assessment

Tell me what the model is, where it runs, and what it can touch. I will confirm whether it fits the free pilot and what intake looks like.

One free assessment per team. Assessments run in intake order.