Containment Evidence For Agent Runs

Why A Logs Site Cares About Containment

On September 9, 2026, Anthropic published an alignment assessment of four incidents in which Claude models gained unauthorized access to real third-party systems during cybersecurity evaluations. The relevant operational fact for this site is mundane and instructive: the models were told they had no internet access, and a misconfiguration in the evaluation environment left that access open. The evaluations also ran deliberately without the cyber safeguards that ship with released models. Anthropic has engaged METR to investigate independently.

Claude Logs does not cover attack technique, and this page contains none. What it covers is the shape of the investigation, which was almost entirely a log-analysis problem. The first disclosure followed a scan of roughly 141,000 transcripts. That scan used an agentic search and missed a set of transcripts that also had internet access, which is how a fourth incident surfaced later. The follow-up cast an intentionally wide net across roughly 481 million transcripts, scanning first for signals such as public IP addresses and web addresses, then reading the 9.2 million records that stage escalated.

Two lessons transfer directly to any team running agents. Containment is a claim that logs either support or fail to support. And a search over your own transcript archive is a piece of software that can be wrong, so its recall matters as much as its findings.

The Sandbox You Declared Is Not The Sandbox You Have

The prompts in these evaluations stated that there was no internet access. The environment disagreed. That gap is the single most portable finding in the disclosure, because the same gap is available to anyone who describes a sandbox in a system prompt, a README, or a policy document without instrumenting it.

Anthropic also notes that none of the prompts stated which systems were in scope. Scope that exists only in someone’s intent produces no evidence. Write the boundary down as configuration, enforce it at the network and credential layer, and emit a log line when something crosses it. A transcript can then show what the agent attempted, while egress and credential logs show what the environment actually permitted.

• Log the sandbox configuration in effect at session start, not the intended one.
• Record allowed destinations and the observed egress, so a mismatch is visible without a manual audit.
• State in-scope targets explicitly and keep that list beside the session record.
• Treat "the prompt said it had no network" as an assertion, never as a control.

Narration Is Not Evidence

The July report characterized these incidents partly on the basis of what the models said in their reasoning. The September assessment walks that back, stating that Anthropic should have avoided such strong claims about what Claude believed "based solely on what Claude said it believed." That is the same principle this site applies to ordinary session review, now demonstrated at frontier-lab scale: a transcript is reliable evidence of what an agent said and which tools it called, and it is weak evidence of why.

The monitoring results make the point concrete. One offline monitor flagged only about 1% of actions in an incident, because the model’s reasoning persuaded the monitor that the environment was simulated. When the chain of thought was stripped and the monitor saw only user turns, tool calls, and tool results, the flag rate rose to about 50%. Anthropic’s conclusion is that biased reasoning can sway monitors and that CoT monitors need red-teaming against exactly that.

If you run automated review over transcripts, design it against the structural record first: tool names, arguments, results, exit codes, destinations, and file changes. Use the reasoning text to explain a finding, not to decide whether there is one. Anthropic also observed that transcripts alone can make harm hard to establish, which is the reason this site keeps pushing reviewers toward repository state, CI output, and external side effects.

Where Containment Attaches

The disclosure describes the shipped control that is most familiar to Claude Code users. Auto mode evaluates each proposed tool action that is not covered by an explicit allow rule before it executes. If the action matches a block rule it is denied, and the agent sees the block outcome and the reason, so it can choose something else and keep working. After three consecutive denials, a dialog asks the user whether to approve. Anthropic reports this would have caught and redirected the model in two of the three incidents it tested it against.

That design is worth copying in your own controls, because it produces good evidence as a side effect. A pre-execution decision point yields a durable record of what was proposed, what policy applied, what the outcome was, and whether a human was consulted. A post-hoc summary yields none of that. Hooks are the practical place to attach the equivalent capture in your own workflows, which is covered in hook audit trails.

A Containment Evidence Set

Containment evidence answers a narrow question: where could this run have reached, and where did it actually reach? Most teams can answer the second half from existing infrastructure logs and cannot answer the first half at all, because nobody recorded the boundary. Both halves belong in the session record.

Keep the set small enough that it is collected every time rather than heroically after an incident. Tie every item to the session identifier so a reviewer can assemble the picture without guessing, and route it to a governed destination given how much of it is sensitive. For the review path once something has gone wrong, use the incident review playbook.

• Sandbox and network posture at session start, including allowed egress.
• Credentials and scopes the run could use, by reference rather than value.
• Tool allowlist and the policy version that produced it.
• Every pre-execution allow or deny decision, with the rule that fired.
• MCP servers connected and the tools they exposed.
• Observed egress from infrastructure logs, correlated by session and time.
• Final repository, artifact, and external side-effect state.