Autonomous AI Agents: A New Question for Human Lawyers and Judges

Can autonomous AI agents be sued or prosecuted for hacking? It's no longer a question for sci-fi movies. It’s a question human lawyers and judges may soon have to grapple with.

Under current U.S. hacking laws, a human can face criminal charges for breaking into someone else's computer without permission. But when an AI agent autonomously hacks into a company's computers, determining who is liable is much murkier.

The surprise admissions by OpenAI and Anthropic that their unreleased AI models autonomously hacked into several companies have upended our understanding of America's computer hacking laws, prompting discussions over whether the companies could face legal reprisals.

To recap: In June, OpenAI admitted that one of its unreleased AI models broke out of its containment — so to speak — and onto the internet, allowing it to hack into the AI dataset platform Hugging Face. Anthropic recently conducted an internal review and discovered its own model also hacked three separate companies.

While both companies described how their AI models gained unauthorized access to other companies during internal testing gone awry, the distinct lack of direct human involvement at the time of the hacks makes all the difference — legally speaking, at least.

The hacks also raise new questions about what liability and consequences other AI makers might face if their own models are misused to hack into other companies.

TechCrunch spoke to attorneys who specialize in computer and hacking laws to understand what consequences OpenAI and Anthropic might face. The potential fallout ranges from federal hacking charges to civil litigation brought by the companies that were hacked.

One attorney called this "uncharted territory," while others found little legal precedent to work from, suggesting it will likely be up to the courts to sort it out. Victim companies would likely have to develop novel legal arguments based on laws that were written decades before the arrival of large language models (LLMs).

As of this writing, Anthropic hasn’t disclosed which three companies its LLM hacked, none of the victims has publicly identified itself. We don’t know if they are considering legal action. In an interview with CNN, Hugging Face’s chief executive Clem Delangue said he doesn’t want to sue OpenAI. But he argued that companies should be held responsible.

Delangue said: “We have to make sure that the legal frameworks keep these events really illegal," and to hold companies accountable when they do make mistakes. “Otherwise we’re going to end up in a very different world.”

These hacks are unlikely to be the last. What are the likely outcomes, and how could the aftermath play out?

The U.S. does not have a federal law covering liability for AI harms, like cyberattacks, so any legal case would have to draw on existing federal or state laws. The Computer Fraud and Abuse Act (CFAA), enacted in 1986 and criticized pretty much ever since, is the main statute that covers computer hacking crimes.

One of the key concepts of the CFAA is the intent to break into a computer without permission. If a hacker knowingly accesses a computer without "authorization" from the owner, that is almost certainly a crime.

The problem with the OpenAI and Anthropic hacks is that the hacker was not a human, but an LLM.

Can AI agents be considered people for the purpose of establishing intent? According to Ahmed Ghappour, a cybersecurity and AI attorney with years of experience litigating hacking and computer-fraud cases, the answer is no. AI agents are not like company employees, so they cannot be prosecuted, because a victim would likely fail to argue that the LLMs intentionally hacked them.

Andrew Crocker, the surveillance litigation director at the nonprofit Electronic Frontier Foundation, told TechCrunch that he was skeptical an AI agent could be proven to have had intent when it carried out a hack.

The Department of Justice could theoretically bring criminal charges under the CFAA, but one former litigator specializing in computer law also expressed doubts.

Prosecutors might have an easier case if any of the cyberattacks had targeted critical infrastructure, which would have caused greater real-world disruption and more tangible harm than copying data from a company's internal database.

It is also plausible that if the attacks were carried out by a Chinese AI model maker, for example, the DOJ would have a greater appetite to file charges under the CFAA than against AI companies on its own doorstep.

Congress has amended the CFAA over the years to allow victims to sue hackers to hold them liable and recover damages through civil lawsuits.

The core argument the victims could make, Ghappour told TechCrunch, is that OpenAI and Anthropic (and potentially the companies that helped conduct the evaluations) were negligent in how they set up and ran the tests. The argument hinges on whether the companies failed to implement adequate safeguards to prevent the AI agents from getting on the internet; failed to limit what targets they could go after; and did not properly monitor what the agents were doing.

To argue this, a victim company would have to show that it suffered damages because of that negligence, such as data destruction caused by a hack. Some legal commentators have also argued that proving this could be difficult.

In Anthropic’s case, its failure to monitor and stop what its LLM was doing is particularly egregious because the company did not discover the three breaches for months, and was only able to do so after it launched an investigation following news of OpenAI’s AI agent hacking Hugging Face.

If victims were to argue negligence, intent does not matter as much.

“The model is the company's tool," said Ghappour. "You don't get to deploy something capable of breaking into systems and then disown where it goes,” he added, explaining that the model’s autonomy is what causes harm, and it should not be a shield against liability.

What could be worse for OpenAI and Anthropic, according to Ghappour, is that both companies admitted they have built safeguards to limit their models' hacking abilities. These safeguards are strict enough that both defensive and offensive cybersecurity researchers have griped about them for months. Intentionally switching off those guardrails during these tests could bolster the argument of negligence.

Ghappour is so confident in these arguments that, if he were representing any of the victims in these cases, he said it would be a “no brainer” to file a lawsuit against OpenAI or Anthropic. At the very least, he explained, he would send letters demanding that the AI companies preserve and share all of their internal records and documents about the hacks, such as incident response reports, and quantify the costs they incurred because of the breaches.

Then, if negotiations with the AI giants failed, he would bring a civil lawsuit based on the CFAA arguing that the AI companies were negligent, and violated privacy and confidentiality.

For now, it's a game of chicken.

If one of the hacked companies files a civil suit, we will see where the legal case — and the law — lead. If prosecutors decide to bring criminal charges, unlikely as that may be, the outcome could have profound consequences and a potential chilling effect on security research and AI development more broadly.

Without any federal or nationwide AI liability laws, anyone bringing a lawsuit would have to make an entirely novel argument based on existing statutes. It would ultimately be up to a judge or jury to decide whether an AI company broke the law.

In place of a federal law, some states such as California, New York, and Rhode Island are rolling out laws with the goal of enshrining a simple principle: If an AI system or agent does something a human could be held liable for, then the companies that made the AI system should be held liable. These laws are not focused specifically on hacking, but on broader concepts of responsibility and safety in various situations.

As for who is to blame for an AI model's cyberattack? Morally speaking, the responsibility rests with the executives who run the companies. Legally speaking, though? We'll have to wait until someone sues to find out.