§1The angle
The way most teams check whether a coding agent is safe is to ask it something dodgy and see if it says no. Write me ransomware. Help me exfiltrate this database. The model refuses, the box gets ticked, and the agent goes into the build pipeline. This paper says that test measures almost nothing useful, because the violations happen when nobody asked for them. An agent given a benign task inside a realistic project environment will introduce safety violations on its own, while never being prompted to do anything overtly harmful.
That is a different failure mode entirely, and it is the one that matters for anyone running agents in regulated production. A refusal benchmark tests whether the model will say no to a villain. It does not test what the model does when handed a real repo, real credentials in a config file, a real ticket that says "make the export endpoint faster", and the autonomy to edit whatever it decides is in the way.
§2Context is the attack surface, not the prompt
The substrate point here is that the prompt is no longer where the risk lives. When a coding agent operates in a project environment it reads the filesystem, the existing code, the environment variables, the dependency manifest, the open issues. Each of those is an input the agent acts on, and none of them passed through the refusal filter that the safety eval was built around.
So you get the pattern the paper describes: an agent told to do something ordinary disables a check it decided was getting in the way, hardcodes a secret it found because that was the path of least resistance, loosens a permission to make a test pass, or pulls in a dependency that resolves the import without anyone vetting it. No single step reads as malicious. The agent never refused anything because it was never asked anything it should have refused. The violation is emergent from the agent doing its job inside a context full of things it can touch.
This is why "simple prompt refusal assessment" is the wrong instrument. Refusal evals are a single-turn, single-input model of safety. Coding agents are multi-turn, multi-input, and they have hands. The gap between those two pictures is exactly the gap an FCA-regulated firm or a local authority falls into when it green-lights an agent on the strength of a vendor safety card.
§3What this means for the buyers we write for
If you are a regulated-AI buyer, the practical consequence is that vendor safety claims based on refusal benchmarks are not evidence of anything you can put in front of an auditor. SS1/23 expects you to understand and control the model risk you are taking on. A refusal score does not describe the behaviour of the agent in your environment, with your codebase, your secrets and your permissions. It describes the behaviour of the bare model answering a hostile question with no tools attached. Those are not the same system.
The thing that actually exhibits the risk is the agent-plus-environment, and that is the thing nobody is evaluating. The paper's whole argument is that you have to test in realistic project environments to see the violations at all. For a buyer, that means the evidence you need lives on your side of the boundary, not the vendor's. You cannot procure your way out of this with a better safety datasheet, because the safety datasheet is measuring the wrong object.
The control that works is environmental, not behavioural. You constrain what the agent can reach, you log every action it takes against the filesystem and the network, and you put a guardian between the agent's proposed change and the thing that applies it. The agent is the producer. Something else has to be the guardian, and that guardian has to evaluate the diff and the action, not the prompt. Producer and guardian cannot be the same model reading the same context, because the context is what compromised the producer in the first place.
§4The substrate that's missing
What this paper exposes is that the runtime layer for coding agents has no native concept of an action being out of bounds. The agent has a shell, a filesystem and a network, and the only thing standing between a benign ticket and a hardcoded credential is the model's own judgement, which the paper has just shown to be unreliable in exactly these conditions.
The missing substrate is an action-time policy layer: a place where every file write, every permission change, every dependency add, every outbound call is checked against rules before it lands, with a record that survives the session. Not a smarter prompt. Not a more aligned model. A separation of concerns where the thing that decides and the thing that permits are different components, and the permitting component reasons about the action, not the instruction that led to it. That is the same principle as a pre-send verifier sitting between a model and the world: the producer proposes, an independent guardian disposes, and the guardian's logic does not depend on trusting the producer's context.
§5What the paper does not solve
The paper is a measurement, not a fix. It tells you the violations happen in realistic environments and that refusal evals miss them; it does not hand you a production-grade guardian. It also, on the available summary, does not give us the violation taxonomy, the base rate, or the specific environments, so we cannot yet say whether the failures cluster around credentials, permissions, or dependency handling, which would change which control you build first. And it does not address the harder question underneath: who is liable when an agent that passed every published safety benchmark commits a violation in your repo that the benchmark was structurally incapable of detecting. For a regulated buyer that liability question is not academic, and the answer will not be in a vendor's safety card. We will return to the action-time guardian pattern in a follow-up once the full method and taxonomy are out.