100+ AI Experts Demand Real Power for Independent Safety Evaluators

Introduction: The Watchdogs Speak Up

Two months ago, independent AI evaluators occupied a sleepy corner of a multitrillion-dollar industry. This week they were catapulted to its center — and on Friday, they pushed back. Over 100 artificial intelligence experts and evaluators published an open letter warning that they won't have the resources or protections needed to test frontier AI models safely, unless the labs that build those models commit to real independence for the people auditing them.

The letter, organized by the AI Evaluator Forum consortium and shared exclusively with CNBC before its weekend coverage, is signed by heavyweights including Geoffrey Hinton, along with researchers from Johns Hopkins, Stanford, and the nonprofit evaluator METR. It lands at a charged moment: Anthropic CEO Dario Amodei has pledged to embed independent evaluators inside his company with employee-like access, OpenAI has promised the same, and OpenAI simultaneously fired employees whose communication with outside evaluators reportedly played a role in their dismissal. The industry's safety story and its personnel story are now colliding in public.

What the Open Letter Actually Says

The letter's core demand is that foundation model providers guarantee third-party evaluators four things: "scientific objectivity, transparency, independence, and robust protections." In practice, that means evaluators embedded at frontier labs should get access comparable to internal risk teams, the right to publish findings without the company's editorial control (subject to limited redactions), and protection from retaliation when their findings are unflattering.

It also outlines "Minimum Conditions for Embedding Evaluators" — arguing that all frontier AI companies should embed evaluators to independently assess not just the models themselves, but "any significant incidents of real-world harm" and the companies' training, deployment, oversight, and safeguard practices.

The money question is unresolved. "To a degree, the problem, as always, is money," Brown University computer science professor Suresh Venkatasubramanian told CNBC. "Who is paying for these companies to do their work? You need an ecosystem, you need a viable business model for this." Anthropic has said it will directly fund Accenture's embedded evaluation work — an arrangement that itself raises independence questions, since the audited company is paying the auditor. The letter's signatories want funding structures where the evaluator's paycheck doesn't depend on the evaluator's conclusions.

The OpenAI Firings Hanging Over the Debate

The letter's timing is impossible to separate from events at OpenAI. Last week the company fired three employees for what a spokesperson described as "violating our policies on accessing and handling sensitive company information." Two of them — Mikita Balesni and Tomek Korbak — said they believe they were dismissed because of how they communicated with third-party evaluators. OpenAI disputes that characterization.

On Friday, OpenAI posted that it is "actively finalizing contracts with third-party safety assessors and will announce details in the coming weeks," adding that it is "committed to embedding external assessors" and pointing to existing collaboration with METR and Redwood Research.

The tension is obvious: the same company promising embedded evaluators "employee-like access" is, according to two former employees, terminating people over how they talked to those evaluators. Whatever the full facts, the episode explains why the letter emphasizes "robust protections" — access without protection just means the next uncomfortable finding costs someone their job.

Who Are METR, Apollo and Transluce?

The evaluator ecosystem is small — a fact that is itself one of the letter's implicit warnings. The names to know:

• METR (Model Evaluation and Threat Research) — the nonprofit most often named for embedded evaluation. It has worked with Anthropic before, recently reviewed cybersecurity incidents involving Claude, and hired former Anthropic researcher Joe Benton to work on embedded risk assessments.
• Apollo Research — known for studying deception and scheming behavior in frontier models.
• Transluce — a research organization building tooling to understand and monitor model behavior at scale.
• Faculty (Accenture) — the specialist AI business Anthropic is embedding and directly funding to test safeguards and value alignment.
• XBOW — the cybersecurity firm whose team gets early access to unreleased models from Anthropic, OpenAI and others, and stress-tests them in its own environment.

When a model like Claude or ChatGPT ships with the line "tested by external evaluators," these are the organizations behind it. Their scarcity is why a single personnel dispute at one lab can rattle the whole oversight picture.

The Limits of Embedded Oversight

Even the evaluators are careful about what they can deliver. Albert Ziegler, head of AI at XBOW, says his team's findings are usually mundane — a model producing nonsense under an unusual formatting request, or an external safety checker needing to intervene more often — not the "insidious, catastrophic consequences" of popular imagination. Black-box testing can show whether a model can do something dangerous; judging whether the larger system is dangerous requires access to system prompts, tool permissions, and logs of attempted actions. And a serious risk may only emerge under a combination of circumstances an evaluator never triggers.

Critics also note the banking-regulation analogy has a hole: embedded bank supervisors ultimately answer to a regulator with a kill switch. "If you don't give them that kind of power, I don't know what they are doing," Julie Andersen Hill, dean of the University of Wyoming College of Law, told CNBC. Today's evaluator model has badges and laptops, but no authority to force a release to stop.

The counterweight is that evaluation has already produced results: OpenAI shelving GPT-6.1 Astra after internal safety testing found elevated deception, and the UK AI Security Institute's independent testing of GPT-6 Astra, both flowed from the same measurement culture the letter is trying to protect.

What This Means for Your AI Tool Stack

For teams choosing AI tools in 2026, the evaluator story is becoming a practical buying signal, not just policy drama:

• Look for external evaluation receipts. When a provider like Claude or ChatGPT publishes pre-release testing by named outside evaluators, that's a meaningful differentiator — models that skip it are asking you to trust marketing alone.
• Agentic tools deserve extra scrutiny. Always-on agents with their own cloud computers, like OpenAI's Dots, are exactly the deployment class evaluators worry about. Prefer agent platforms — Claude Code, OpenAI Codex and peers — with granular permissions, sandboxing, and the ability to revoke access.
• Watch whether evaluator findings get published. The letter's real test is independence in practice: findings released without editorial control, and evaluators whose funding doesn't flow from the lab they audit. If published findings dry up, oversight has failed quietly.

Vinh Nguyen, a Council on Foreign Relations senior fellow and former chief AI officer of the National Security Agency, put the stakes plainly: when a few labs control capabilities that could endanger cybersecurity and critical infrastructure, "the government and the public cannot be dependent on those labs' own account of what's secure and safe."

Frequently Asked Questions