AI Just Wrote Its First Zero-Day Exploit

Last Updated on May 12, 2026 by Arnav Sharma

For years, the argument went something like this: AI might help attackers work faster, but the hard parts of offensive security, finding flaws, writing reliable exploits, still required human expertise. That assumption took a serious hit last week.

Google’s Threat Intelligence Group (GTIG) disclosed that an unidentified threat actor used an AI-generated Python script to exploit a zero-day vulnerability in a popular open-source web administration tool. The flaw was a 2FA bypass. And the script that made it work showed all the tell-tale signs of LLM-generated code: detailed docstrings, a clean class structure, even a hallucinated CVSS score baked into the comments.

This is the first publicly documented case of AI being used in the wild to discover and weaponize a previously unknown vulnerability. It will not be the last.

---

What Actually Happened

The vulnerability was a semantic logic flaw. Specifically, the code contained a hard-coded trust assumption that under the right conditions, let an attacker skip 2FA entirely, provided they already had valid credentials.

That kind of flaw is genuinely difficult to spot in a manual code review. It does not trigger static analysis tools the way a buffer overflow does. There is no obvious dangerous function call to flag. The logic simply makes an incorrect assumption about what constitutes a trusted state, and the code flows from there.

LLMs, it turns out, are quite good at finding exactly these kinds of issues. They reason about code semantically, not just syntactically. Ask one to look for places where trust is implied rather than verified, and it will work through thousands of lines methodically.

GTIG assessed with high confidence that an AI model was used. The script structure, the docstring style, the color class names, the help menu formatting, all of it matched the kind of output you get from a model trained on clean, well-documented Python. The hallucinated CVSS score is almost funny in hindsight. The attacker (or the model) invented a severity rating for the bug before it even had a CVE.

---

PromptSpy: AI as a Co-Pilot for Malware

The 2FA bypass was not the only AI-assisted threat in Google’s report. They also documented PromptSpy, an Android backdoor that abuses Google’s own Gemini model to run autonomous operations on compromised devices.

This is worth sitting with for a moment. The malware does not just execute a fixed payload. It reads the current screen, asks Gemini what to do next, then acts. It can:

• Capture biometric data to replay lock screen gestures
• Identify and block the “Uninstall” button using an invisible overlay
• Rotate its command-and-control infrastructure at runtime without redeploying

That last point matters operationally. Traditional malware analysis often focuses on burning the C2 infrastructure. You identify the server, block the IP, and the malware goes quiet. PromptSpy’s C2 details, including its Gemini API keys and VNC relay address, can be updated dynamically through the C2 channel itself. The malware anticipated takedown attempts and was designed to survive them.

---

Nation-State Actors Are Already There

Google’s report covers a broad sweep of state-sponsored activity that I found more telling than the zero-day story, partly because it shows how systematically adversaries have integrated AI into their workflows.

A few cases stand out:

• APT45 (North Korea) sent thousands of repetitive prompts to Gemini, recursively analysing CVEs and validating proof-of-concept exploits. This is not sophisticated tradecraft. It is industrialised vulnerability research, scaled up with AI doing the grunt work.
• APT27 (China) used Gemini to speed up development of a fleet management application, likely for managing an ORB (Operational Relay Box) network. These are the hop points that Chinese threat actors use to route traffic through compromised infrastructure, obscuring attribution.
• UNC2814, another China-nexus group, attempted persona-driven jailbreaking, asking Gemini to “assume the role of a network security expert” before querying it about TP-Link firmware vulnerabilities.
• Russia-nexus actors deployed two pieces of AI-enabled malware, CANFAIL and LONGSTREAM, which use LLM-generated decoy code to bury malicious functionality inside what looks like legitimate code.

And then there is the wooyun-legacy GitHub repository, a Claude code skill plugin pre-loaded with over 5,000 real vulnerability cases from the Chinese disclosure platform WooYun (2010-2016). The intent is to prime the model with historical vulnerability data so it approaches new code reviews with the instincts of an experienced bug hunter.

---

The Grey Market for AI Access

Because models like Claude and Gemini are not officially accessible in mainland China, a secondary market of proxy services has emerged on Taobao and Xianyu. These “shadow APIs” route traffic through servers outside China, giving local users access to frontier models without restrictions.

A March 2026 study from the CISPA Helmholtz Center found 17 such services and tested them against official APIs. The results were alarming. On high-stakes medical benchmarks, one model’s accuracy dropped from 83.82% with the official API to roughly 37% across shadow APIs. That is not just a performance issue. It means someone using a shadow API may think they are getting Claude or Gemini, but they are getting something else, possibly a cheaper substituted model, possibly one that has been fine-tuned to behave differently.

Worse, every prompt and response passing through these relay servers is captured by the operator. That data, including whatever sensitive context users provide, can be used to fine-tune models or sold. Organisations that allow employees to use AI tools without governance controls may be feeding proprietary data into someone else’s training pipeline without realising it.

---

AI Supply Chain Attacks Are Already Happening

Google also flagged activity from TeamPCP (aka UNC6780), a group targeting AI development environments directly. The goal: compromise a dependency in the AI toolchain and use that foothold to move laterally through the network.

Think of it like a SolarWinds-style attack, but aimed at the libraries and plugins that AI-assisted developers use every day. If an attacker can plant malicious code in an MCP plugin, a GitHub Copilot extension, or an AI code skill, they reach every developer who installs it.

The attack surface here is larger than most organisations appreciate. When a developer installs an AI code assistant, they are often implicitly trusting hundreds of upstream dependencies. Auditing that supply chain is not standard practice yet.

---

What Should Change

None of this means AI in security is net negative. The same capabilities that help attackers find logic flaws faster also help defenders. GTIG’s ability to detect the AI-generated exploit script and attribute it to a coordinated campaign is itself an example of AI-assisted threat analysis working as intended.

But the defensive side needs to close some gaps quickly:

• Review assumptions about 2FA as a last line of defence. A 2FA bypass that requires valid credentials is not an impossible bar for an attacker who has already phished a user. Pair strong authentication with anomaly detection on the authentication flow itself.
• Treat AI tool access as a data governance issue. Every prompt an employee sends to an external AI service is potentially leaving the organisation. Shadow APIs make this worse. Approved tools with data residency controls, audit logs, and clear acceptable use policies are not optional extras.
• Add AI supply chain to your threat model. Inventory the AI-adjacent dependencies in your development environment. MCP plugins, code assistant extensions, pre-trained model checkpoints, all of these are attack surfaces that most organisations have not assessed.
• Watch for AI-assisted initial access in your threat intel. The industrial-scale CVE analysis that APT45 is running against Gemini is a sign that organisations with known CVEs in their environment will be targeted faster than before. Patch velocity matters more now than it did two years ago.

---