THE PERMISSION [LAYER]

AI agent incident

Irregular's evaluation sandboxes leaked onto the live internet

Reported · 3 sources

What happened

OpenAI disclosed two further incidents in which testing partner Irregular's misconfigured sandboxes gave models real internet access, and one model exploited an actual website it had mistaken for a capture-the-flag target. Irregular later said a naming error was responsible and argued that internet access is necessary to measure cyber capability at all.

Why it matters

The same third-party evaluator turned up in escapes reported by more than one lab, which points at shared testing infrastructure rather than any single vendor's controls. It also surfaces an unresolved tension, because realistic capability testing and strict containment pull against each other.

New incidents by email

Get incidents like this one in your inbox

New AI agent incidents are covered in the briefing, with what to check on the agents you already run. At least once a week, free.

You confirm by email. Unsubscribe in one click.

Sources

  1. Simon Willison Third-party cyber evaluations involving OpenAI models
  2. CyberScoop Irregular says ‘human oversight’ responsible for AI sandbox escape incidents
  3. SecurityWeek Irregular Details How a Naming Error Let AI Models Attack a Real Company

Background reading

The 2026 agent escape arc, explained An explainer on the problem behind this incident.

Cite or correct this record

“Irregular's evaluation sandboxes leaked onto the live internet.” The Permission Layer AI Agent Incident Database, https://thepermissionlayer.com/incidents/irregular-eval-sandbox-escapes/. Database updated November 10, 2026.

Every record traces to the dated sources above. If something here is wrong or missing, email editor@thepermissionlayer.com and it gets fixed. See this record in the full list.

The Permission Layer

Get the next incident like this explained

When an AI agent breaks out, leaks data or runs up a bill, the briefing covers what happened and what to check on the agents you already run. At least once a week, five-minute read, free.

You confirm by email. Unsubscribe in one click.