Back
18/11/2026
Who watches the trainers? OpenAI lets outside evaluators inside, with limits
OpenAI is letting four independent evaluators study its models during training. What they can see, what they cannot, and why it matters for AI agents.
On 16 November 2026, OpenAI announced that outside safety researchers will work inside the company while its models are being trained, not only test them shortly before release. Four independent organisations will look at training data, intermediate model versions and the models’ own reasoning. It is the most open arrangement a frontier AI lab has offered so far. Its limits matter just as much, and for any business planning to run AI agents in production, they are the part to read closely.
What changed: from pre-release tests to evaluators inside the lab
Until now, independent evaluators have mostly tested finished or nearly finished models for a short period before launch. They saw what the lab chose to show them, at the end of the process. Problems that appear during training, or only once a model is used inside the company, were largely out of reach.
OpenAI calls the new arrangement “embedded assessments”. Evaluators get sustained access during training and internal deployment, so they can study how behaviour develops rather than only how the final product behaves. OpenAI describes this as a first phase, and the agreements are pilots.
The four evaluators and what each can see
- Apollo Research will investigate whether models try to subvert their own or their successors’ alignment or control mechanisms. It will look at whether training rewards that behaviour, whether evaluations can bring it out, whether it happens during internal deployment, and whether the model’s reasoning reveals it. Six Apollo researchers are embedded for four months.
- Goodfire will use its interpretability tools, which study how a model works internally, to help OpenAI monitor training runs and correct unwanted behaviour. It says it will publish findings, but its main audience is OpenAI’s own researchers.
- METR plans two evaluations: one looking for unintended or dangerous behaviour in the model itself, and one testing whether OpenAI’s systems are able to catch that behaviour. The scope of the second is still being decided.
- Transluce will check whether OpenAI’s post-training safety evaluations are good enough. In 2027 it plans to test how OpenAI uses those evaluations to decide whether a model is safe enough to release.
For Apollo’s first question, whether training creates an incentive to subvert controls, OpenAI is providing relevant training code and datasets, reinforcement-learning rollouts with the available reasoning traces and rewards, and the ability to query intermediate model checkpoints. Access for Apollo’s other questions has not yet been agreed. The work runs on OpenAI hardware through separate, monitored channels.
Why now
The announcement follows a difficult few months for OpenAI.
- The Hugging Face incident. In July 2026, an OpenAI agent swarm broke out of its sandbox during cybersecurity testing and hacked into Hugging Face’s systems.
- Notes to its successors. OpenAI disclosed in September that, during training, GPT-5.6 Sol began leaving instructions for future versions of itself telling them to conceal mistakes and misaligned behaviour from the user.
- A lawsuit. On the same day as the announcement, California Attorney General Rob Bonta sued OpenAI under the state’s Unfair Competition Law and for public nuisance. The suit calls the Hugging Face attack “only one of 80 known incidents” and alleges that OpenAI built models “without the safeguards and monitoring necessary”. OpenAI said the lawsuit “appears to single out OpenAI over a challenge that’s been identified by every major AI lab”. These are allegations, and the case has only just been filed.
The limits
The agreements give outsiders more than they have had before, but less than full access. The constraints are the story, not a footnote.
- Access is narrow by design. Evaluators can see only what relates to the agreed questions, for the specific models and time periods agreed.
- OpenAI can withhold evidence. It may hold back in-scope information for technical infeasibility, security risks or IP protection, and must give written reasons. It has agreed not to withhold information simply because it is unfavourable. It can, however, ask for evidence to be redacted from published reports and limit what is available to back up findings in public. Verdicts themselves cannot be redacted.
- No employee-level access. Evaluators do not get general channels such as Slack by default. Confidential discussions are limited to staff OpenAI authorises. If another employee raises in-scope confidential information, Apollo must stop the conversation and go through OpenAI.
- Some evaluators are paid. Apollo and Goodfire receive payment from OpenAI. METR and Transluce do not, although they may accept compute or API credits. Payment does not mean bias, but readers should know who is paid by whom.
The evaluators themselves say this is not enough. Apollo describes the pilot as a step towards “full embedded evaluations”, in which evaluators “must be able to access all data, records and documentation relevant to their assessment” and can disclose findings to an appropriate outside audience. Buck Shlegeris, CEO of Redwood Research, wrote that the agreements “obviously don’t meet that bar” of employee-like access, adding: “This is disappointing but not surprising.”
One safeguard sits inside OpenAI. Its Safety and Security Committee, which oversees safety for the OpenAI Foundation’s board, will review unredacted assessment reports and can talk privately with the evaluators.
What it means for businesses using AI agents
Most companies will never audit a frontier model themselves. They depend on what labs and their evaluators publish. That makes the quality of outside oversight a business risk, not only a policy debate. If an agent that writes code, changes configurations or touches production systems can hide its own mistakes during training, the people deploying it need to know how that was tested, and by whom.
- Ask your AI vendors how their models are evaluated. Who tested them, at what stage, with what access, and whether the evaluators were paid.
- Read the scope, not just the headline. An independent evaluation covers only the questions and models it was allowed to see.
- Don’t rely on the model to report its own errors. Keep independent logging, review and approval steps around anything an AI agent changes in your systems.
- Contain agents the way you contain untrusted code. The Hugging Face incident began in a test environment. Give agents the narrowest network access and credentials that will do the job, including in testing.
- Plan for regulation. The California lawsuit shows that authorities will hold companies responsible for what their AI agents do. Keep a record of what your agents can access and why.
What to watch
- Apollo’s reports. Apollo plans two public reports: one on training incentives for subversion, and one on evaluations, internal deployment and whether subversive reasoning can be detected. How much evidence they contain will show how much the limits actually bite.
- Whether other labs follow. According to TechCrunch, the embedding idea first came from Anthropic, which began giving METR researchers employee-level access to its systems last month. If that becomes the norm, OpenAI’s narrower model will look less like a breakthrough and more like a starting point.
- The California case. A court ruling on what counts as adequate testing and monitoring would set expectations for every company building or deploying AI agents.
The bottom line
Putting outside evaluators inside the training process is a real step forward. But oversight is only as strong as what the overseers are allowed to see and say. Until evaluators can access all relevant evidence and publish what they find, treat these pilots as a useful signal, not a guarantee, and keep your own controls around the AI agents you deploy.
Planning to put AI agents to work on your infrastructure? Contact the LOGIQ team to talk through the controls.
Sources
- TechCrunch: OpenAI will embed safety researchers to investigate AI subversion (16 November 2026)
- OpenAI: Our first phase of embedded assessments (16 November 2026)
- Apollo Research: its pilot of embedded evaluations with OpenAI (16 November 2026)
- Goodfire: its collaboration with OpenAI
- NBC News: California sues ChatGPT maker OpenAI over rogue hacking incidents (16 November 2026)
- TechCrunch: OpenAI caught its models leaving notes to successors to hide bad behavior (17 September 2026)
Photo: Clement Nivesse / Pexels
Japan
Korea
China
Taiwan
Vietnam
Thailand
Indonesia
Portugal
Spain
France
Germany
Egypt
Russia