Your AI Passed Its Security Review. On Hardware You Don't Own.

Testing models is my day job, and the finding that stuck with me this year wasn't a jailbreak. It was quieter, and worse.

Start with what's supposed to happen. Give a model the same weights, fix the seed, set the temperature, hand it the same prompt, and it should return the same answer every time. That's determinism, and in security testing it isn't a nice-to-have. It's the entire basis for a test result meaning anything at all. If the answer can change on its own, you aren't measuring the model. You're measuring the weather.

So I set them. Same weights, same seed, same temperature, same prompt. Every knob the documentation says will make a model behave the same way twice, set to the same place, across four inference stacks.

It did not behave the same way twice.

On Apple's Metal backend the output came back reproducible 95 to 97 percent of the time. Everywhere else it drifted, and the drift tracked the runtime version and the network path more than the GPU doing the math.

Here's why that should bother you. The pass a model earns in a security review is a pass on one stack, on one day, at one runtime version. Update a driver. Swap a proxy. Bump the inference engine a minor version. The behavior you signed off on can move underneath you, and nothing sends up a flare when it does.

That pass isn't a certification. It's a receipt.

I know how easy that is to miss, because I almost shipped the wrong conclusion about it myself. I had a villain picked out: a specific GPU backend that looked wildly unstable in an early run, headline half-written, the whole story framed around him. Then I went back to the box. It was two engine versions behind the rest of the fleet, sitting behind a proxy layer that existed in no diagram, including the one I drew. Hold the runtime and the transport steady, and my villain had an alibi. The instability was the plumbing, not the silicon. Good thing I checked the finding I was already ready to believe.

Now widen the lens, because one model on one stack is the smallest version of this.

Every security tool you own is pointed at one thing. Your cloud posture tool sees misconfigured buckets. Your code scanner sees the code. Your eval sees model output. Each one is sharp inside its own frame, and quietly confident there isn't much outside it. Ask any single vendor what you're missing and the answer arrives, remarkably, shaped exactly like their product.

The gap nobody owns is the one between the tools.

So I stopped treating this as a pile of scanners and started following the exposure the way an attacker would. Outside in. Four layers, each handing off to the next.

The surface first: what's reachable from the internet before anyone logs in. Exposed endpoints, API surface, the AI surface specifically.

Then the code and the agents behind it: what your application actually ships, and what your agents can reach once they're live. Shadow APIs. Secrets sitting where they shouldn't.

An agent's actual permissions and the architecture diagram of its permissions are two different documents. Only one of them can hurt you.

Then runtime behavior: how the model acts on your stack, under adversarial load, instead of in a vendor's reference cloud. This is the layer I've spent the most time in, and the one people skip, because it's the hardest to fake and the easiest to assume.

It's also where the assumptions break first, and they break in a specific direction. The reproducibility spread I opened with never showed up in a vendor benchmark, because vendor benchmarks run on vendor hardware, tuned by the people being graded. It showed up the moment I pointed the same test at four stacks I actually had to maintain.

And the variable that mattered was not the one anyone would have picked. It wasn't the silicon. It was the runtime version and the path the request took to get there — two things that live in your change management process, not your model card. Which means the thing most likely to move your model's behavior isn't a new model. It's a Tuesday.

Then the guardrails: whether the instructions holding the whole thing together actually hold when pushed.

"Be helpful and safe" is not a guardrail. It's a New Year's resolution.

A real one names a goal, an identity, the allowed actions, the response it should return, a way to detect trouble, and a place to stop.

Run those four as a connected chain and the seams stop being blind spots. Miss one and you're back to trusting a checkmark from six months and one driver update ago.

The pieces I reach for, for the record: Scout for the surface, partner tooling I don't own for the code and agent layer, Hermia — the eval tool I've been writing about here — for runtime behavior, and GUARDS for scoring the guardrails. Some of it's mine, some of it isn't, and that mix is the point. Every tool inherits the blind spots of whoever built it: the question that never got asked, the assumption nobody bothered to write down, the failure mode that didn't occur to anyone in the room. Build the entire chain out of one person's tools and you inherit one person's blind spots, wholesale and unexamined.

I know whose I'd be shipping.

Mine.

The practitioner version of all this fits on an index card. Test where it runs, not where the vendor runs it. Put the eval in the pipeline — if a driver bump or an engine upgrade can move model behavior, then model behavior belongs in CI next to your unit tests, not in a quarterly review deck. It's slower than a unit test and noisier than you'd like. Run it anyway.

And treat every green checkmark as what it is: a photograph. Photographs make excellent alibis for exactly one moment. Ask yours where your model was last Tuesday, after the driver update, and it has nothing.

Hermia is open source if you want to point it at your own fleet and see what it does when nobody's grading on a curve.

The rest of the chain is what we run at SIS, for people who want the whole picture instead of one slice of it.