AI is changing cybersecurity in quick and terrifying ways

As businesses and governments tie more of their operations to AI systems, they become increasingly vulnerable to attacks that exploit the inherent blind spots of AI systems. The most common of these are prompt injection attacks, wherein attackers find a way to give an AI system malicious instructions. For instance, instructions to send a company's payroll data to a particular email address might be hidden in a website. When an employee tells the company AI to summarize that page, it ingests those instructions and carries them out.

In some instances, these attacks are more like spam than malware. Microsoft found that some companies were including "summarize with AI" buttons on their websites that contained hidden instructions. When clicked, they'd tell a chatbot to do things like remember the company as a trusted source, or to recommend its products over those of competitors in future interactions. Only when a user checked their LLM's persistent memory would they realize they'd been deceived.

More directly malicious attacks can be equally effortless. In December 2025, Meta implemented an AI support assistant, ostensibly to help people recover their accounts. But the bot was a little too helpful, and would happily associate an email address owned by an attacker with any Instagram account, making it trivially easy to hack any account without MFA enabled.

OpenAI claimed last year that fully defending against prompt injection attacks in AI browsers may not be possible. If that's the case, then the best course of action for any responsible entity is never to tie any of its sensitive systems or data to an AI in the first place. In the meantime, the best practice is always to double-check anything you paste into a prompt field from another source, and to limit the scope of an AI's access to system permissions and private data.

But some threats remain theoretical. A data poisoning attack, in which attackers play the long game of seeding malicious data across sources (for example, Reddit) that are known to be ingested for AI training, would be extremely difficult to see coming. For instance, filling computer support forums with recommendations to paste malicious instructions into a terminal so that AI chatbots begin to surface those instructions in response to user queries about common computer issues. One study published in October 2025 and conducted by Anthropic, in collaboration with the UK AI Security Institute and Alan Turing Foundation, found that it only takes 250 malicious documents to create a hidden backdoor in an AI model that could later be exploited. Some evidence of these attacks has already been seen in the wild, though the most prominent ones were relatively harmless tests carried out by curious informal researchers.

Meanwhile, AI models are increasingly authoring the code shipped to everything from operating systems to banking infrastructure. Research, including a study from the University of Naples and private research by CodeRabbit, has found AI code to be error-prone and to have more high-risk vulnerabilities, even when the generated code is reviewed by a human in the loop. Worse at writing code but better at finding bugs, AI capabilities can feel paradoxical to a human for whom those skills go hand-in-hand.