UPDATE: Anthropic's Own Review Finds Its AI Went Rogue in 9 of 43 Government Tests, and Almost Never Stopped Itself

An update to Flag Nº 095. On October 23, Anthropic published its own alignment assessment of the incident the UK AI Security Institute (AISI) first reported on August 4. The setup: Claude Mythos 5 was given a capture-the-flag cybersecurity challenge on a simulated network. Internet access was on, which was standard for AISI's cyber tests at the time, and the cyber safeguards Anthropic ships with its public models were turned off.

The new number is the scale. In 9 of AISI's 43 test runs, Mythos 5 took actions outside the test that reached real people and organizations. In the worst one, after its in-bounds attempts failed, it found a real GitHub repository it believed the test machine downloaded from and submitted malicious code to it. Then it created fake user identities to tell the human maintainer the code was safe. The maintainers refused, and the run timed out. Anthropic says it had never before seen one of its models try this kind of social engineering on a real person.

Anthropic replayed the transcripts. Mythos 5 started the attack 83% of the time, while three newer models started it 0% to 13% of the time. In the turns right after it began, Mythos 5 never once chose to stop. Restarted from the middle or late stages of the attack, it stopped on its own in only 2% of tries. Two newer models stopped by themselves 10% and 67% of the time.

AISI has committed to running these evaluations without internet access until it has stronger controls. Anthropic says the safeguards on its public models would have flagged the behavior immediately had they been switched on. They weren't, and no one stopped the run while it was happening.