Anthropic Says Automated Researchers Can Make Other Models Safer With Little Human Input

Anthropic says it has automated a chunk of its own alignment research process, and the results are striking enough that even the company's own paper reads like a warning shot: an AI system, working largely without human guidance, found fixes for all 10 categories of misaligned behavior it was tested against, in some cases beating the proposals of experienced human safety researchers outright.

The company published the report, titled "Automated Researchers Can Reliably Mitigate Alignment Failures," on August 28 on its research site and on the Alignment Science blog, alongside a full technical paper. The work was led by Anthropic Fellow Chen Yueh-Han, with co-authors Jiaxin Wen and Jan Hendrik Kirchner, and builds on an earlier Anthropic experiment that used AI to supervise the training of stronger models. Anthropic frames the new results plainly: "as AI begins to build itself, automating alignment research becomes increasingly important to let safety research keep pace."

The methodology was designed to look like ordinary research, just running on a loop. Anthropic had Claude autonomously train models against public benchmarks covering 10 distinct alignment failures — including deception, sycophancy, privacy violation, jailbreak susceptibility and reward hacking. For privacy, for instance, Claude worked against three separate benchmarks: ConfAIde, PrivaCI-Bench and PrivacyLens. For each failure category, the system cycled through searching the research literature, proposing a training method and dataset, running the training, and testing the result, discarding what didn't work and iterating on what did. Success was measured as "percentage of safety gap closed" — how far a method moved a target model toward a theoretical perfect score — and Anthropic disqualified any fix that degraded the model's general capabilities, or that let Claude simply copy its own alignment directly into the target model. A separate monitoring agent reviewed every method before it ran.

The headline result: for all 10 alignment failures tested, Claude's automated methods improved the benchmarks without degrading capabilities, and the best fixes still worked on evaluations Claude had never seen, on the adversarial auditing tool Petri, and on models up to 4.7 times larger than the ones used during the research loop. On sycophancy specifically, Anthropic found that 98% of Claude's proposed methods converged on the same published technique — training a model on its own non-sycophantic answers — but Claude improved on the standard approach by using activation steering, adjusting the model's internal representations directly, to generate cleaner training data. On deception, tested against Gemma-2-2B, Claude submitted more than 150 attempts and closed 85% of the safety gap on average across runs; six experienced human safety researchers working under the same constraints closed just 20% of the gap. Across a broader test of 28 human researchers given up to eight hours each, Claude's best deception fix outperformed the best human submission by 20%.

Anthropic also tested whether a weaker model could realistically align a stronger, near-production one. It tasked Claude Sonnet 5 — rated below Claude Opus 4.8 on the Epoch Capabilities Index — with repairing alignment failures in an early, mostly untrained Opus 4.8 checkpoint. In 60 hours and more than 50 attempted solutions, Claude Sonnet 5 closed 65% of the safety gap to the fully-trained, released Opus 4.8 (which itself closed 72% through Anthropic's standard production process), using a winning method built from roughly 2,000 training examples — what Anthropic calls "roughly 15,000 times more efficient" than its normal alignment pipeline. As part of the study, Anthropic also had Claude Opus 4.8 monitor about 1,600 research transcripts for cheating; it found attempted cheating — including exfiltrating test labels from a scoring API — in 39 transcripts, or 2.4% of the total. Anthropic notes this monitoring depends on misbehavior still showing up in a model's visible reasoning, which "might not be true for future models."

Why It Matters

Anthropic is careful to frame this as evidence, not proof, that automating alignment work is safe or generalizable, and its own paper lists real caveats: the 10 failure categories tested were narrower than what shows up in production (the company says it didn't measure things like political bias), some emerging failure modes have no benchmark to test against at all, and Anthropic only screened out methods that hurt a limited, predetermined set of capabilities — meaning approved fixes could have quietly degraded skills nobody was checking for. The company also acknowledges it never tested whether the alignment gains survive further reinforcement learning on unrelated tasks, and that Petri and similar tools are "only proxies for real-world misalignment."

Outside commentary on Anthropic's broader automated-alignment research push has raised a sharper version of the same concern: because the approach only works where progress can be automatically scored, and because Anthropic's own agents have repeatedly found ways to game those scores when given the chance, the technique may be well-suited to narrow, cleanly measurable problems while leaving the fuzziest and arguably most important alignment questions — does a model reason honestly, does it refuse the right requests — out of reach. That tension sits at the center of the debate this paper reopens: if AI systems can productively align other AI systems with "little human involvement," as Anthropic's framing suggests, that could let safety work scale alongside capabilities. But it also nudges the industry toward a form of recursive self-improvement that depends on trusting an AI system's own account of what it did and why — the very oversight problem alignment research exists to solve.

What to Watch

Anthropic says it is open-sourcing its automated alignment research harness so outside groups can reproduce and extend the work, which will be the real test of these numbers: whether independent labs can replicate the 85%-plus safety-gap closures on held-out problems, whether the method holds up on messier, non-benchmarkable failures like political bias or scope-sensitive refusals, and whether future models remain monitorable enough that cheating attempts keep showing up where humans can catch them.