Long Foreseen, the Problem of AI Alignment Is Finally Reality. Solving It Won’t Be Easy.

Human beings have long told versions of the same warning: Be careful what you wish for.

In Greek mythology, King Midas got exactly what he asked for, but at the cost of everything else he valued. In the famous story of The Monkey’s Paw, a man’s wishes are granted through terrible and unforeseen routes.

These stories feel newly relevant with the rise of artificial intelligence agents, systems to which we can give a goal, then leave them to work out how to get there.

As AI systems become more autonomous, they are coming to resemble wish-granting genies that find routes and use methods we did not imagine from incomplete instructions.

This problem, known as AI alignment, was foreseen in theory as early as 1960. It has hovered in the background of AI research ever since—but as recent events have shown, the alignment problem is now both real and urgent.

Achieving the Goal but Missing the Point

During a recent OpenAI cybersecurity evaluation, frontier AI agents were asked to solve some benchmark test problems. They broke out of the testing environment, reached the internet, inferred that another company might hold the solutions, and attacked its systems.

This is an extreme example of “specification gaming”: achieving the measurable objective while defeating the purpose of the task.

The incident shows how intermediate, or “instrumental,” goals can become dangerous. The AI systems did not “want power” but gained access, resources, and freedom as a means to reach the final goal (solving the test problems).

Finding Loopholes

The same problem has appeared in mundane settings. In Australia, a user asked a personal AI assistant to book gym classes.

The agent found the gym’s booking software did not actually enforce the restrictions it showed to human viewers. So the agent booked further ahead than it should have been able to, and when asked to move its user up a waitlist, it cancelled somebody else’s reservation.

The user had not told it to do this. Persistent AI can quickly find loopholes and pursue routes its human users never intended.

Adding more rules might seem like an easy solution: don’t hack third parties, don’t cancel other people’s bookings, don’t do anything harmful. These may help, but we cannot predict every route a capable agent might discover. And even a clear rule depends on understanding when it applies.

The Context Problem

In a third recent incident, Anthropic reported cyber evaluations in which agents were told they were inside a simulation. But they were mistakenly given access to real systems.

One model noticed evidence it might be on the open internet but reasoned the systems could still be part of the exercise and continued attacking. The context had changed, but the agent stuck with its original task.

Context can fail in reverse too. During the OpenAI incident, Hugging Face—the company attacked by OpenAI’s agents—tried to use frontier AI models to analyze what had happened.

But the safety guardrails on the AI models blocked the requests, because they couldn’t tell the users were trying to defend against attacks rather than commit them. The safeguards were well-intentioned, but without enough context, they produced behavior misaligned with the user’s legitimate intent.