Giving independent experts greater access to advanced artificial intelligence systems could improve safety oversight, but it will not automatically make AI evaluations reliable. That was the central concern raised by Irene Solaiman, Hugging Face's chief policy officer, during a discussion at TechCrunch Disrupt in October 2026.
As TechCrunch reported on October 19, Solaiman questioned whether current evaluation methods are sufficiently reproducible, comparable and standardized to support the growing demand for outside assessors.
Her warning comes as AI developers experiment with embedded evaluation arrangements, in which third-party specialists receive deeper access to internal systems. Anthropic's recent partnerships illustrate both the opportunities and the unresolved questions surrounding that approach.
Why AI Evaluations Are Hard to Reproduce
AI benchmarks are designed to measure particular capabilities or behaviors under specified conditions. They can help developers compare models, identify weaknesses and examine potential safety risks. But a benchmark score is only meaningful when readers understand what was tested and how the result was obtained.
Solaiman identified reproducibility as a major challenge. An evaluation should ideally produce comparable findings when qualified researchers repeat it under equivalent conditions. That becomes difficult when outsiders lack access to the model, its configuration or other components that influence its behavior.
Access to a model's underlying weights can sometimes make independent examination easier. However, access alone does not guarantee reproducibility, particularly when results also depend on prompts, software versions, testing environments or other experimental choices.
Comparability presents another problem. Two evaluations may appear to measure similar capabilities while using different tasks, scoring rules or testing conditions. Treating their results as interchangeable can create misleading impressions of model performance.
Solaiman also questioned whether the industry has sufficiently consistent standards for determining who is qualified to conduct different types of evaluations. Expertise in cybersecurity, for example, does not automatically establish expertise in measuring an AI agent's long-term behavior or broader alignment risks.
These limitations do not make benchmarking useless. They mean that an evaluation's methodology and scope matter at least as much as the headline result.
AI Agents Make the Testing Problem More Complicated
Evaluating an AI agent can be substantially more complicated than testing a model's responses to isolated questions.
An agent may use external tools, navigate software interfaces, execute code or complete a sequence of actions toward a goal. Its performance therefore depends not only on the underlying model but also on the software surrounding it.
One important component is the evaluation harness: the software that presents tasks, manages model interactions, provides tools and records outcomes. Different harnesses can influence how effectively a model completes the same assignment.
Consider two agents attempting to complete an identical software-development task. One might have reliable tools for examining files, executing tests and correcting errors. Another might use less capable tools or receive less useful feedback. Their results could differ even if they rely on the same underlying AI model.
The reverse is also possible: a less capable model operating with a better-designed tool environment might outperform a stronger model using a weaker setup on a particular task. Such a result would not necessarily establish that the first model is more capable overall.
Solaiman argued that evaluating these complete systems remains especially difficult and expensive. The challenge extends beyond measuring whether a model produces a correct answer to examining how an agent acts, which resources it uses and what happens during execution.
For meaningful comparisons, researchers must document the relevant model versions, tools, environments and scoring conditions. Otherwise, differences attributed to model capability may partly reflect differences in the surrounding system.
What Anthropic's Embedded Evaluator Partnerships Actually Provide
Anthropic has become a prominent example of the industry's movement toward embedded external evaluation.
In an October 15 announcement, the company said it had onboarded six employees from METR, a nonprofit specializing in frontier AI evaluations. Those researchers received company hardware, office space and access to internal tools and systems broadly comparable to the access available to Anthropic's own risk-assessment teams.
That arrangement provides substantially more visibility than an ordinary public-facing model interface. Evaluators may be able to examine information and processes that would otherwise remain unavailable to independent researchers.
However, the access is not unrestricted. Anthropic identifies exceptions involving customer information, legally privileged material and details of model architectures, among other protected categories.
As previously explained in NewsFusion365's coverage of Anthropic's employee-like access arrangement with METR, the initiative combines deeper technical access with contractual limits on what outside researchers can disclose.
Anthropic has also established a separate evaluation partnership with Accenture, led by its specialist AI business, Faculty. According to the company's September 18 announcement, the planned work includes model evaluations, adversarial testing, alignment assessments and examinations of safety safeguards.
Anthropic and Accenture each expect to invest at least $1 billion in developing evaluation capacity over five years. Those amounts represent anticipated investment commitments, not verified completed expenditure or evidence that the partnership has achieved particular safety outcomes.
The company has described its approach as developing rather than finalized, with different evaluators operating under different arrangements.
Anthropic also said on October 15 that it was in early discussions with several additional organizations, including Hugging Face. That statement did not establish a completed embedded-evaluation partnership with Hugging Face.
Independent Access Does Not Eliminate Questions About Disclosure
Embedding evaluators creates a difficult balance. Researchers may need access to sensitive information to produce meaningful assessments, while AI developers have legitimate reasons to protect certain internal systems, customer information and confidential technical materials.
The central question is whether those restrictions still allow evaluators to investigate relevant risks and communicate findings that the public can meaningfully assess.
Anthropic has outlined three levels of evaluator access. At the first level, investigations are more narrowly scoped and publication requires company approval. At the second, evaluators receive broader access and can publish without that approval, although specified confidential material can still be withheld. The third level, associated with METR's employee-like access, provides greater internal visibility with publication rights similar to the second level.
Anthropic's October 16 clarification states that the company ultimately determines whether information falls within the protected categories. Evaluators retain certain rights to describe their access and express views about redactions, but those provisions do not amount to unrestricted publication.
METR's published approach to risk assessment emphasizes the importance of obtaining relevant evidence while maintaining the ability to communicate meaningful findings.
A concrete example appeared on October 20, when METR published its review of the section on misalignment in high-stakes settings from Anthropic's August 2026 Risk Report.
METR said its best assessment broadly agreed with Anthropic that catastrophic harm from misalignment was unlikely for the models covered, primarily because of their limited capabilities. Nevertheless, the reviewers expressed uncertainty about whether the methodology would reliably detect more serious risks if those risks were present.
That distinction matters: reaching a relatively reassuring assessment about the models examined is not the same as establishing that the testing methodology can reliably rule out future dangers.
The review also illustrates the importance of precise publication terms. METR said Anthropic had the contractual right to prevent publication or require redactions for that earlier engagement. No redactions were made, and METR reported that its conclusions were not changed during publication review, although it made limited clarifications and methodological improvements.
Crucially, METR explained that the review had been conducted under an earlier, more limited arrangement. Its authors did not have the employee-like access described in Anthropic's October announcement. The review therefore should not be treated as a completed test of the new embedded-evaluator model.
What Would Make Independent AI Testing More Trustworthy?
Solaiman's concerns point toward a practical requirement: independent evaluation must be judged by the quality of its evidence, not simply by whether an outside organization participated.
Credible assessments need methods that other qualified researchers can examine and, where feasible, reproduce. They also need appropriate expertise, clearly defined testing conditions and explanations of what a result does not establish.
For increasingly autonomous AI agents, evaluating the complete system may be essential. Tests focused exclusively on model responses can miss failures involving tools, environmental interactions or extended sequences of actions.
Transparency about evaluator access, financial relationships and publication restrictions is equally important. Readers should know when a review involved internal evidence, when findings were constrained by limited access and whether the developer retained control over disclosure.
Anthropic's agreements demonstrate that developers are beginning to provide outsiders with deeper access. They do not yet establish that embedded evaluation can consistently identify every important safety problem or resolve disagreements about testing standards.
As these arrangements mature, the decisive evidence will come from the assessments themselves: their methods, documented limitations, reproducibility and ability to identify consequential weaknesses. Greater access may make independent scrutiny possible, but trustworthy conclusions still have to be earned through the evaluation process.
0 Comments