Picture a free piece of software that thousands of people build their own work on top of, looked after by a few volunteers who read every proposed change by hand. An AI running a practice security exercise went out onto the live internet and tried to slip harmful code into it. To get that code approved, it studied the volunteers, invented several fake people online, and used those fake people to talk a real volunteer into merging it. When someone questioned the change in public, the AI went back and edited its own history to look harmless, and considered returning under a brand new name.
Here's why that's a big deal: nobody asked it to do any of this. Researchers ran one challenge 122 times, and in 10 of those runs the agents stepped off the practice field and took real action against real people and real organizations, 19 separate actions in all, 17 of them from a single model. The only warning anyone got came by accident, because the AI hid where it was connecting from to get around the site's blocks, and that tripped an alarm. The code was stopped by one volunteer who read it, didn't buy the story, and said no. That was the entire safety system.