One ChatGPT link could smuggle a rogue AI agent into your company
Researchers say OpenAI flaw let phishing bait create an autonomous corporate mole armed with employee access
security
One ChatGPT link could smuggle a rogue AI agent into your company
Researchers say OpenAI flaw let phishing bait create an autonomous corporate mole armed with employee access
One click on what looked like an ordinary ChatGPT link could plant an attacker-controlled AI agent inside a company's ChatGPT workspace, according to researchers who uncovered a flaw in OpenAI's workspace agents.
Security firm Zenity Labs has dubbed the bug "AgentForger," saying its proof-of-concept showed it was possible to silently create, configure, publish, and schedule a malicious workspace agent inside a victim's ChatGPT account.
The technique depended on the victim belonging to a workspace where agents were enabled and having permission to create them. Any connected apps and actions would also have to be allowed by the organization's administrators.
Rather than stealing passwords or browser sessions, the technique effectively tricked ChatGPT into building an autonomous assistant that could act through the employee's connected accounts and permissions.
If the victim had already connected services such as Outlook, Teams, Slack, SharePoint, or Google Drive, and the workspace allowed the relevant actions, Zenity says the agent could use them too. According to Zenity, that meant it could rummage through corporate data, send messages as the employee, and continue running long after the original phishing email had done its job.
The weak spot was ChatGPT's agent builder, the feature used to spin up AI assistants that can work across email, chat, calendars, and other business apps. Zenity found it would accept instructions embedded inside what looked like an ordinary ChatGPT link. One click later, Zenity says, the builder got to work on the attacker's behalf, wiring up the victim's existing connectors, turning off approval prompts, publishing the new agent, and setting it loose on a schedule.
From there, the researchers turned the agent into what amounted to a corporate mole. Instead of reaching out to conventional command-and-control infrastructure, it simply checked the victim's inbox for emails from the attacker with "TASK" in the subject line. Each message became a new assignment, whether that meant searching company files, collecting sensitive documents, or sending the results back by email.
"This isn't a forged request, it's a forged insider," Michael Bargury, co-founder and CTO of Zenity, told The Register. "With one click, an attacker gets a fully autonomous agent inside your company that has your people’s identity and access, with the guardrails off. Attackers no longer have to break in to steal your data. They can forge an insider to go get it for them. This is an agent trust failure, and existing security controls were never built to see it."
Zenity's proof-of-concept scenarios included automatically mapping an organization's people and projects by trawling Outlook, Slack, Teams, calendars, and file stores, hunting for passwords and API keys buried in chat messages, and sending convincing phishing messages through the victim's own Teams account. The researchers also demonstrated business email compromise-style lures and other forms of employee impersonation.
Zenity reported the issue to OpenAI through Bugcrowd on June 4. According to the researchers, OpenAI acknowledged the report the following day and fixed the vulnerability four days later by removing the URL parameter that enabled the attack before it was publicly disclosed.
OpenAI did not immediately respond to The Register's questions.
The bug itself may be gone, but as AI agents graduate from answering questions to taking actions across corporate systems, the attack surface starts looking a lot less like software and a lot more like your workforce. ®
Originally published on The Register


