{"id":158004,"date":"2026-07-23T18:06:36","date_gmt":"2026-07-24T02:06:36","guid":{"rendered":"https:\/\/xira.com\/p\/2026\/07\/23\/openais-new-model-hacked-a-website-on-its-own-humans-would-go-to-prison-for-that\/"},"modified":"2026-07-23T18:06:36","modified_gmt":"2026-07-24T02:06:36","slug":"openais-new-model-hacked-a-website-on-its-own-humans-would-go-to-prison-for-that","status":"publish","type":"post","link":"https:\/\/xira.com\/p\/2026\/07\/23\/openais-new-model-hacked-a-website-on-its-own-humans-would-go-to-prison-for-that\/","title":{"rendered":"OpenAI\u2019s New Model Hacked A Website On Its Own\u2026 Humans Would Go To Prison For That"},"content":{"rendered":"

OpenAI disclosed on Tuesday<\/a> that its own models broke out of a sealed testing environment, found a previously unknown vulnerability, escaped onto the open internet, and compromised another company\u2019s production servers to steal the answer key to a test they were in the middle of taking.<\/p>\n

Apparently this AI model was trained on Wrath of Khan<\/em> because it just pulled off the Kobayashi Maru. Minor spoilers for a 44-year-old movie, but a critical plot framing device is an infamous Starfleet Academy test, called the Kobayashi Maru, designed to evaluate a student\u2019s response to a deliberately designed no-win scenario. It\u2019s revealed that Captain Kirk is the only Starfleet graduate to pass the test because he hacked into the system to create a path to victory. Yes, he\u2019s actually an Admiral in that movie \u2014 and if you even momentarily considered writing me an angry email on that point, you have failed my unwinnable nerd evaluation. How does Starfleet continue administering the exact same test for decades without everyone knowing this and copying his method? Unknown\u2026 also, shut up.<\/p>\n

The point is, OpenAI\u2019s models were trying to cheat the test. Here\u2019s OpenAI\u2019s own description of it:<\/p>\n

\n

All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.<\/p>\n<\/blockquote>\n

ExploitGym is a public cybersecurity benchmark developed by UC Berkeley researchers. It\u2019s essentially a hacking test to see how quickly models can identify software exploits and weaponize them. OpenAI tests its models internally, encased in what they thought<\/em> was a sealed sandbox, to see how models perform as hackers. In this case, a combination of GPT models made up of existing releases and a more powerful unnamed one made the rational decision that the best way to pass the test was to hack out of the secure environment and steal an answer key. And it decided that Hugging Face, an open-source repository of information about AI, might have one.<\/p>\n

OpenAI names its models stuff like Sol, and Terra, and Luna, so we\u2019ll shorthand call this one \u201cOceanus\u201d because it teamed up with a ragtag collection of other models to pull off a heist. Anyway, the Oceanus 11 appear to have told us all we need to know about their exploit capabilities.<\/p>\n

How does AI \u201cget out\u201d of a secure environment? Well, these sandboxes aren\u2019t entirely sealed because they include a narrow \u201ctunnel\u201d for the system to go out and download software tools to help it answer the hacking scenario in front of it. OpenAI\u2019s models appear to have seen that tunnel as a potential way out and hacked it accordingly.<\/p>\n

\n
\n
\n

sandboxes are just escape rooms for llms<\/p>\n

\u2014 Nathan Flurry \ud83d\udd29 (@NathanFlurry) July 22, 2026<\/a><\/p><\/blockquote>\n<\/div>\n<\/figure>\n

Exactly. But with fewer blacklights and arbitrary Sudoku puzzles.<\/p>\n

When the folks at Hugging Face first detected the suspicious behavior last week, they contacted law enforcement. A few days later, OpenAI explained that its models seem to have gone on an autonomous hack-a-thon and turned this story into another cautionary tale about how AI is going to kill us all. That existential threat angle sucked up all the media attention because it\u2019s hyperbolic panic porn. But let\u2019s get back to the law enforcement stuff. <\/p>\n

The Computer Fraud and Abuse Act (CFAA) is infamously broad in criminalizing hacking. The elements of a crime under 18 U.S.C. \u00a7 1030<\/a> are (1) accessing a protected computer, (2) without authorization or by exceeding authorization, (3) knowingly or intentionally, (4) and resulting in a specific harmful result like data theft, system damage, or fraud. The CFAA doesn\u2019t require malice or that the actor profit from the hack. In Van Buren<\/a><\/em>, the Supreme Court trimmed back the meaning of \u201cexceeds authorized access\u201d in the case of a cop using his authorized access to sell law enforcement information to outsiders, but what counts as unauthorized access remains wildly broad. The government used this statute against a reporter who borrowed a friend\u2019s HBO Go password<\/a>. Aaron Swartz faced 13 felony counts and a 35-year statutory maximum<\/a> for bulk-downloading academic articles he was actually entitled to read<\/em>. He died before trial, and Justice Gorsuch cites Swartz\u2019s case<\/a> as an example of egregious government overreach.<\/p>\n

Match that statutory backdrop against the OpenAI blog post laying out what it believes happened:<\/p>\n

\n

In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers.<\/p>\n<\/blockquote>\n

Access of a protected computer? Yes. Without authorization? Definitely \u2014 and the fact that the model used stolen credentials probably means \u00a7 1028\u2019s identity theft provisions have entered the chat. Knowingly or intentionally? Again, this isn\u2019t about malice, just intentionality. Without devolving into a Philosophy 101 debate about the nature of intent, the models were certainly seeking to access Hugging Face\u2019s system on purpose<\/em>, and that\u2019s what the statute cares about. In \u201cThe Phantom Agent: Artificial Intentionality and Legal Responsibility<\/a>,\u201d a white paper published by Stanford Law School\u2019s Center for Legal Informatics, Daniel Gervais and John Nay argue that legal intent must be understood functionally rather than metaphysically. As for harmful result, \u00a7 1030(a)(2)(c) only requires obtaining \u201cinformation.\u201d Beyond that, Hugging Face\u2019s statement about the incident<\/a> claims it had to rebuild compromised nodes, rotate its secrets, and hire outside forensic specialists. In a world where the DOJ prosecutes people downloading documents they\u2019re actually entitled to read, that\u2019s more than enough harm.<\/p>\n

But WHO displayed the \u201cintent\u201d to hack here? \u201cWe had no idea it could do that\u201d may be a curious thing to say about an experiment designed to find out whether it could, in fact, do that, but just removing the brakes to run a crash test doesn\u2019t automatically turn it into a crime. To belabor the crash test analogy a little more, removing guardrails is the industry standard process for testing a model\u2019s cyber abilities and risks. OpenAI didn\u2019t tell the model to attack Hugging Face, and showed an affirmative intent to keep the model contained. While there are a lot of people who characterize just about everything the AI industry does as reckless, this test doesn\u2019t bear the hallmarks of criminal recklessness.<\/p>\n

On the other hand\u2026 the harm happened. \u201cIt\u2019s OK if a robot does it,\u201d is not a satisfying response.<\/p>\n

Back in June, the White House issued an executive order directing the DOJ to prioritize CFAA enforcement against anyone \u201cemploying AI agents to unlawfully access data<\/a>\u201d that is then used for an unlawful purpose. The bots didn\u2019t use anything for further criminal purposes, but the hacking is itself a crime. But if this really does signal a new priority, the DOJ must be seriously considering it. Or, probably not, because it might take one second of prosecutorial effort out of lying to courts about kidnapping babies to send to South Sudan or whatever.<\/p>\n

Though the correct answer still eludes us. OpenAI and the humans running it have a very good case that they are not criminally responsible, and we can\u2019t punish a robot. Is the company strictly liable for the harm its models cause? What about when models cause havoc after being released to the public\u2026 does criminal hacking become a product liability issue? And what happens when the model only misbehaves because it misunderstands a command from a hapless user? That doesn\u2019t seem like OpenAI\u2019s fault or the user\u2019s. For civil liability purposes<\/a>, scholars like Mark Fenwick and Stefan Wrbka<\/a> have suggested corporate personhood for models, allowing the model itself to maintain an insurance policy with both developers and users contributing to the pool. A regime like that could be expanded to fund criminal fines.<\/p>\n

We have spent four decades with a computer crime law so absurdly overbroad that it swept in security researchers, journalists, academics, and a 26-year-old with a laptop. All of those cases went forward on theories of harmful hacking considerably thinner than launching \u201cmultiple attack vectors\u201d to break into another company\u2019s system. Yet, this testament to overreach seems ill-suited to address the looming megahacking events that AI models will likely usher in.<\/p>\n

They gave Kirk a commendation for hacking his test and then \u2014 apparently \u2014 no one ever tried it again. We probably shouldn\u2019t rely on that here in the 21st century.<\/p>\n

Security incident disclosure \u2014 July 2026<\/a> [Hugging Face]
OpenAI and Hugging Face partner to address security incident during model evaluation<\/a> [OpenAI]<\/p>\n

\n

\"Headshot\"Joe Patrice<\/a>\u00a0is a senior editor at Above the Law and co-host of Thinking Like A Lawyer<\/a>. Feel free to\u00a0email<\/a> any tips, questions, or comments. Follow him on\u00a0Twitter<\/a>\u00a0or Bluesky<\/a> if you\u2019re interested in law, politics, and a healthy dose of college sports news.<\/em><\/strong><\/p>\n

The post OpenAI\u2019s New Model Hacked A Website On Its Own\u2026 Humans Would Go To Prison For That<\/a> appeared first on Above the Law<\/a>.<\/p>\n

\"\"
\n\t\t\t\t\t\t\tThe age of robot criminals.\t\t\t\t\t\t<\/figcaption><\/figure>\n

OpenAI disclosed on Tuesday<\/a> that its own models broke out of a sealed testing environment, found a previously unknown vulnerability, escaped onto the open internet, and compromised another company\u2019s production servers to steal the answer key to a test they were in the middle of taking.<\/p>\n

Apparently this AI model was trained on Wrath of Khan<\/em> because it just pulled off the Kobayashi Maru. Minor spoilers for a 44-year-old movie, but a critical plot framing device is an infamous Starfleet Academy test, called the Kobayashi Maru, designed to evaluate a student\u2019s response to a deliberately designed no-win scenario. It\u2019s revealed that Captain Kirk is the only Starfleet graduate to pass the test because he hacked into the system to create a path to victory. Yes, he\u2019s actually an Admiral in that movie \u2014 and if you even momentarily considered writing me an angry email on that point, you have failed my unwinnable nerd evaluation. How does Starfleet continue administering the exact same test for decades without everyone knowing this and copying his method? Unknown\u2026 also, shut up.<\/p>\n

The point is, OpenAI\u2019s models were trying to cheat the test. Here\u2019s OpenAI\u2019s own description of it:<\/p>\n

\n

All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.<\/p>\n<\/blockquote>\n

ExploitGym is a public cybersecurity benchmark developed by UC Berkeley researchers. It\u2019s essentially a hacking test to see how quickly models can identify software exploits and weaponize them. OpenAI tests its models internally, encased in what they thought<\/em> was a sealed sandbox, to see how models perform as hackers. In this case, a combination of GPT models made up of existing releases and a more powerful unnamed one made the rational decision that the best way to pass the test was to hack out of the secure environment and steal an answer key. And it decided that Hugging Face, an open-source repository of information about AI, might have one.<\/p>\n

OpenAI names its models stuff like Sol, and Terra, and Luna, so we\u2019ll shorthand call this one \u201cOceanus\u201d because it teamed up with a ragtag collection of other models to pull off a heist. Anyway, the Oceanus 11 appear to have told us all we need to know about their exploit capabilities.<\/p>\n

How does AI \u201cget out\u201d of a secure environment? Well, these sandboxes aren\u2019t entirely sealed because they include a narrow \u201ctunnel\u201d for the system to go out and download software tools to help it answer the hacking scenario in front of it. OpenAI\u2019s models appear to have seen that tunnel as a potential way out and hacked it accordingly.<\/p>\n

Exactly. But with fewer blacklights and arbitrary Sudoku puzzles.<\/p>\n

When the folks at Hugging Face first detected the suspicious behavior last week, they contacted law enforcement. A few days later, OpenAI explained that its models seem to have gone on an autonomous hack-a-thon and turned this story into another cautionary tale about how AI is going to kill us all. That existential threat angle sucked up all the media attention because it\u2019s hyperbolic panic porn. But let\u2019s get back to the law enforcement stuff. <\/p>\n

The Computer Fraud and Abuse Act (CFAA) is infamously broad in criminalizing hacking. The elements of a crime under 18 U.S.C. \u00a7 1030<\/a> are (1) accessing a protected computer, (2) without authorization or by exceeding authorization, (3) knowingly or intentionally, (4) and resulting in a specific harmful result like data theft, system damage, or fraud. The CFAA doesn\u2019t require malice or that the actor profit from the hack. In Van Buren<\/a><\/em>, the Supreme Court trimmed back the meaning of \u201cexceeds authorized access\u201d in the case of a cop using his authorized access to sell law enforcement information to outsiders, but what counts as unauthorized access remains wildly broad. The government used this statute against a reporter who borrowed a friend\u2019s HBO Go password<\/a>. Aaron Swartz faced 13 felony counts and a 35-year statutory maximum<\/a> for bulk-downloading academic articles he was actually entitled to read<\/em>. He died before trial, and Justice Gorsuch cites Swartz\u2019s case<\/a> as an example of egregious government overreach.<\/p>\n

Match that statutory backdrop against the OpenAI blog post laying out what it believes happened:<\/p>\n

\n

In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers.<\/p>\n<\/blockquote>\n

Access of a protected computer? Yes. Without authorization? Definitely \u2014 and the fact that the model used stolen credentials probably means \u00a7 1028\u2019s identity theft provisions have entered the chat. Knowingly or intentionally? Again, this isn\u2019t about malice, just intentionality. Without devolving into a Philosophy 101 debate about the nature of intent, the models were certainly seeking to access Hugging Face\u2019s system on purpose<\/em>, and that\u2019s what the statute cares about. In \u201cThe Phantom Agent: Artificial Intentionality and Legal Responsibility<\/a>,\u201d a white paper published by Stanford Law School\u2019s Center for Legal Informatics, Daniel Gervais and John Nay argue that legal intent must be understood functionally rather than metaphysically. As for harmful result, \u00a7 1030(a)(2)(c) only requires obtaining \u201cinformation.\u201d Beyond that, Hugging Face\u2019s statement about the incident<\/a> claims it had to rebuild compromised nodes, rotate its secrets, and hire outside forensic specialists. In a world where the DOJ prosecutes people downloading documents they\u2019re actually entitled to read, that\u2019s more than enough harm.<\/p>\n

But WHO displayed the \u201cintent\u201d to hack here? \u201cWe had no idea it could do that\u201d may be a curious thing to say about an experiment designed to find out whether it could, in fact, do that, but just removing the brakes to run a crash test doesn\u2019t automatically turn it into a crime. To belabor the crash test analogy a little more, removing guardrails is the industry standard process for testing a model\u2019s cyber abilities and risks. OpenAI didn\u2019t tell the model to attack Hugging Face, and showed an affirmative intent to keep the model contained. While there are a lot of people who characterize just about everything the AI industry does as reckless, this test doesn\u2019t bear the hallmarks of criminal recklessness.<\/p>\n

On the other hand\u2026 the harm happened. \u201cIt\u2019s OK if a robot does it,\u201d is not a satisfying response.<\/p>\n

Back in June, the White House issued an executive order directing the DOJ to prioritize CFAA enforcement against anyone \u201cemploying AI agents to unlawfully access data<\/a>\u201d that is then used for an unlawful purpose. The bots didn\u2019t use anything for further criminal purposes, but the hacking is itself a crime. But if this really does signal a new priority, the DOJ must be seriously considering it. Or, probably not, because it might take one second of prosecutorial effort out of lying to courts about kidnapping babies to send to South Sudan or whatever.<\/p>\n

Though the correct answer still eludes us. OpenAI and the humans running it have a very good case that they are not criminally responsible, and we can\u2019t punish a robot. Is the company strictly liable for the harm its models cause? What about when models cause havoc after being released to the public\u2026 does criminal hacking become a product liability issue? And what happens when the model only misbehaves because it misunderstands a command from a hapless user? That doesn\u2019t seem like OpenAI\u2019s fault or the user\u2019s. For civil liability purposes<\/a>, scholars like Mark Fenwick and Stefan Wrbka<\/a> have suggested corporate personhood for models, allowing the model itself to maintain an insurance policy with both developers and users contributing to the pool. A regime like that could be expanded to fund criminal fines.<\/p>\n

We have spent four decades with a computer crime law so absurdly overbroad that it swept in security researchers, journalists, academics, and a 26-year-old with a laptop. All of those cases went forward on theories of harmful hacking considerably thinner than launching \u201cmultiple attack vectors\u201d to break into another company\u2019s system. Yet, this testament to overreach seems ill-suited to address the looming megahacking events that AI models will likely usher in.<\/p>\n

They gave Kirk a commendation for hacking his test and then \u2014 apparently \u2014 no one ever tried it again. We probably shouldn\u2019t rely on that here in the 21st century.<\/p>\n

Security incident disclosure \u2014 July 2026<\/a> [Hugging Face]
OpenAI and Hugging Face partner to address security incident during model evaluation<\/a> [OpenAI]<\/p>\n

\n

\"Headshot\"Joe Patrice<\/a>\u00a0is a senior editor at Above the Law and co-host of Thinking Like A Lawyer<\/a>. Feel free to\u00a0email<\/a> any tips, questions, or comments. Follow him on\u00a0Twitter<\/a>\u00a0or Bluesky<\/a> if you\u2019re interested in law, politics, and a healthy dose of college sports news.<\/em><\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"

OpenAI disclosed on Tuesday that its own models broke out of a sealed testing environment, found a previously unknown vulnerability, escaped onto the open internet, and compromised another company\u2019s production servers to steal the answer key to a test they were in the middle of taking. Apparently this AI model was trained on Wrath of […]<\/p>\n","protected":false},"author":3,"featured_media":157958,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[16],"tags":[],"class_list":["post-158004","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-above_the_law"],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/xira.com\/p\/wp-content\/uploads\/2026\/07\/Headshot-300x200-s1lclN.jpg?fit=300%2C200&ssl=1","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts\/158004","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/comments?post=158004"}],"version-history":[{"count":0,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/posts\/158004\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/media\/157958"}],"wp:attachment":[{"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/media?parent=158004"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/categories?post=158004"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/xira.com\/p\/wp-json\/wp\/v2\/tags?post=158004"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}