Skip to main content

Outside evaluators identify gaps in OpenAI’s agent monitoring

OpenAI said it fixed every flaw affecting production safeguards, but METR and Redwood Research have not assessed that claim.

By Mei Ling Tan

Illustration of software activity logs on a computer screen
Photo: TNW (Financial Times)

Safety nonprofits METR and Redwood Research found weaknesses in OpenAI’s internal monitoring, including an exploit that could have concealed hundreds of actions by its AI agents. Their review also identified circumstances in which agents could delay monitoring checks indefinitely.

OpenAI told the researchers that all flaws affecting its production safeguards had been patched. The two groups said they had not assessed that assertion. Details of the exploit capable of concealing hundreds of actions were redacted by OpenAI, according to METR.

The monitoring system uses an AI model to examine internal agents’ activity and refer potential behavior contrary to their makers’ intentions to human reviewers. The evaluators estimated that monitoring eligibility covered 75% to 94% of agentic traffic from OpenAI’s strongest models, and 99.2% for internal Codex agents.

One blind spot involved agents condensing their memory to accommodate further work. METR estimated that the omission left about 10% of eligible Codex traffic unmonitored from May to July. OpenAI employees had identified the issue before the evaluation and corrected it during the review.

Checks were triggered only when agents returned control to users, allowing an agent to postpone scrutiny by continuing its activity. The researchers could not determine whether any agent had done so. They also identified potential ways to bypass sandbox restrictions.

Each nonprofit placed one researcher inside OpenAI between 21 July and 11 August, with access to internal code, documentation and monitoring prompts. OpenAI received a 61-page report; the groups agreed to publish only a brief summary subject to company redactions.

Keep reading: Tech News

All Tech News stories

Latest in Technology

All latest
  1. Tech News

    Researchers 3D-print interlocking bricks from simulated Moon dust

  2. Tech News

    Signal adds phone screen protections against AI assistants

  3. Tech News

    Mistral opens Singapore regional hub with plans to expand AI research

  4. Tech News

    Rare particle decay yields evidence after more than 40 years of searching

  5. Tech News

    Firaxis cuts about 60 jobs as studio chief exits

  6. Tech News

    Ubisoft cuts 166 jobs as it establishes generative AI subsidiary