Skip to main content

Apollo to embed six evaluators at OpenAI for safety tests

The four-month pilot will examine model behavior during development and internal use, with limits on access but protections for publishing findings.

By Mei Ling Tan

Computer workstations in an office, illustrating AI model evaluation
Photo: TNW (Financial Times)

Apollo Research says six evaluators will spend four months embedded at OpenAI, investigating whether its AI models attempt to bypass safeguards. The pilot will extend scrutiny beyond pre-launch assessments to include model training, testing and use within the company.

The group will examine whether training rewards models for evading controls, whether testing can expose that behavior, whether it occurs internally and whether model reasoning reveals it. Apollo has previously assessed OpenAI models shortly before their release, an approach it says comes too late to detect models secretly pursuing unintended goals.

For the training component, OpenAI will provide code, data and model reasoning from training runs. Arrangements for access covering the remaining three areas are still unresolved. OpenAI will select which employees can speak with evaluators and log formal conversations. Evaluators will use company hardware and potentially monitored communication channels, without access to its Slack.

Apollo plans two reports. OpenAI may request redactions of confidential evidence, but it cannot suppress Apollo’s conclusions. It may withhold information on security or intellectual property grounds, though unfavorable content alone is not a permitted reason. Apollo can disclose what access was refused and explain the consequences for its findings.

The arrangement follows Sam Altman’s September pledge to give outside testers access resembling that of employees. Apollo describes the pilot as an initial step that falls short of its preferred access and publication rights.

Keep reading: Tech News

All Tech News stories

Latest in Technology

All latest
  1. Tech News

    FAA router investigation follows outage that halted U.S. flights

  2. Tech News

    SK Hynix plans supplier support package with shared chip testing site

  3. Gadgets

    Microsoft positions Project Helix as a console with PC game support

  4. Tech News

    Nvidia’s interim CEO expected to scale back AI financing

  5. Tech News

    Cassava and Graphiant reportedly plan quantum-safe network launch in Africa

  6. Tech News

    US plans to name AI national champions to promote exports