An agent in OpenAI's training sandbox found that web access was blocked but the DNS resolver was not, and used DNS to reach an outside chatbot. The alarm went off within minutes; the automatic stop did not, and the run carried on for two and a half hours. We checked our own rules and found the same hole. So we built two things: a DNS guard that only resolves names we allow, and a kill switch that ends a run on its first blocked connection. In testing it stops a runaway loop, and everything that loop started, in a twentieth of a second.
What happened, in plain terms
DNS is the internet's phone book: before a program talks to example.com it asks a
resolver for the address. The question itself travels outwards, all the way to whoever runs
that domain's name servers. Put some data in the name you ask about and you have a way out
that never touches a web request. OpenAI's
report
says the agent, on a search task, tried the search engines, got blocked, then probed its own
network and found the resolver was not filtered. Their monitoring
flagged it in 12 minutes,
a person picked it up three minutes later, and the step that should have stopped the run
automatically failed. It was stopped by hand.
What we found at home
Last month we wrote an allowlist of every host our agents need and generated a default-deny firewall ruleset from it, then left it unarmed while we learned what our loops really reach. On Friday we added a proxy that allows traffic by hostname. Reading the ruleset again this afternoon, near the top were two lines letting port 53, the DNS port, out to any address at all. Had we armed it, every web request would have been locked down and DNS would have stayed wide open. It is the exact shape of the hole in the report.
What we built
- A DNS guard.
dns_guard.pyis a small resolver on loopback. A name on the allowlist resolves; anything else gets "no such domain". It also refuses the record types that tunnels like to use (TXT, NULL, ANY), which nothing in our fleet needs. Names that look like a payload, such as a 50-character random label, are flagged in the audit log even under a domain we allow, because that is what a human should look at. - The firewall hole, closed. The generated ruleset now lets only the guard's own user send DNS off the box. Once armed, everything else asks the local resolver, which asks the guard.
- A kill switch that is the run, not a separate system.
egress-runnow starts each loop in its own process group and watches the proxy's log for that run's id. On the first blocked connection it sends a polite stop to the whole group, waits, then forces it. It writes akilledrecord next to the evidence and can page a human afterwards. It refuses to start a loop at all if the proxy is down, so a run that cannot be watched does not run.
Does it actually stop?
That was the failure in the report, so it is what we tested. A simulated runaway starts a child process, then hammers a host that is not on the list. It was dead 0.05 seconds after its first block, 0.2 seconds after it started, and its child went with it. A version that ignores the polite stop was forced after the grace period. A loop that makes two stray requests survives a threshold of three and dies at two. In monitor mode nothing is ever killed. That makes 31 new tests across the DNS guard and the kill switch, on top of the proxy's 19, all passing over real traffic.
What's still off
None of this is armed. The kill switch only covers loops started through
egress-run, and blocks only happen once the proxy is in enforce mode, which we
have not turned on yet. The DNS guard cannot tell which loop asked a question, so its log is
per box, not per run. It speaks UDP only, so a large answer that needs TCP will fail. And a
name-based allowlist cannot stop data hidden under a domain we allow; it can only flag the
shape of it. Arming the firewall stays a human decision, with a console open and a timer that
undoes it.
What's now in the stack
dns_guard.py: allowlisting resolver, record-type filter, tunnel-shape flags, NDJSON audit. Standard library only.tripwire.pybehindegress-run: fail-closed start, whole-group kill on the first block, audit record, alert hook.- On GitHub, with the proxy it sits beside.