TAG
#evaluations
Stories connected through a shared idea, actor, technology or theme.
Anthropic gives METR employee-like access to evaluate frontier AI risksSix METR employees received company hardware, office space and access to internal tools and systems broadly comparable to Anthropic’s own risk-assessment teams. Anthropic says METR will be able to publish risk assessments without prior approval under the highest planned evaluator-access tier.→Frontier labs face a dispute over independent evaluators’ publication rightsThe companies have publicly supported independent evaluation, but Reuters reported that negotiations include restrictions intended to protect trade secrets and prevent information from moving between labs. Proposed safeguards include lab observation of evaluators and limits on simultaneous work for competing providers.→Langfuse 4.56.0 adds spend alerts and richer agent observabilityVersion 4.56.0 adds spend alerts for ClickHouse-billed organisations and syntax highlighting in session-tool previews. It also revises evaluation-result views, improves agent-session handling and requires date ranges for the observations MCP tool.→OpenAI commits to independent evaluators with employee-like accessAltman said OpenAI agreed that independent evaluators should receive employee-like access and that OpenAI would do the same. He described pacing frontier development as a major topic of recent internal discussions.→Anthropic calls for pacing frontier AI and permanent embedded evaluatorsAmodei said Anthropic would give third-party evaluators ongoing access to its systems, training processes and safety work. He also called for coordination among democratic AI companies and eventually with authoritarian governments.→Amodei clarifies that pacing means verification time, not a moratoriumAmodei said pacing would not halt model training or technical progress. Instead, companies would take more time to align and safeguard models while third-party evaluators verify their claims.→Hugging Face launches Open Alignment InitiativeClement Delangue said the initiative would be led by Hugging Face co-founder Thomas Wolf. The organisation said alignment work should not remain behind the closed doors of a small number of frontier labs.→Anthropic raises its assessed risk for high-stakes misalignmentAnthropic's new risk assessment covers released and internal systems, including Mythos 5 and an unreleased Model 2. It cites recent cyber incidents and declining confidence that existing evaluations reliably capture capability gains.→Anthropic raises its assessed risk for high-stakes misalignmentThe report covers Anthropic's released and internal systems, including Mythos 5 and an unreleased model called Model 2. Anthropic said recent cybersecurity incidents increased uncertainty and that its evaluations may no longer capture capability gains reliably.→