North Korean spies are running local LLMs to cause AI mischief

North Korean government snoops are operating LLMs locally and collecting technology to weave AI into their attack operations, according to South Korean security firm Genians.

The researchers said they observed Kimsuky setting up and operating local LLM environments using Ollama, GPT4All, and Msty, experimenting with other AI tools such as Cursor, and using retrieval-augmented generation (RAG) for local document searches. This prevents the data from getting sucked into the cloud where enemies might see it and try to stop it.

Kimsuky, a cyber-espionage crew that operates under North Korea's Reconnaissance General Bureau, has for years used phishing and decoy documents in attacks targeting government agencies, think tanks, academia and security research organizations. 

Genians’ findings “provide concrete evidence that the Kimsuky-affiliated threat actor is moving beyond one-off experimentation with AI and is continuously preparing to integrate the technology into actual attack capabilities, including malware development, data analysis, and the advancement of attack techniques,” the researchers said in a Monday report.

The North Korean group’s recent phishing emails use ZIP archives containing malicious LNK files - Kimsuky typically disguises these as materials related to international events, research reports, or meeting requests. When the recipient opens the archive and executes the LNK file contained within it, the shortcut runs an embedded PowerShell loader.

In some cases, the goon squad used AI to create lures related to virtual assets and finance, we’re told. These decoy documents “use natural language, a highly polished structure, and formats similar to actual business materials to increase user trust and induce the execution of malicious files,” the security analysts noted.

Additionally, the Pyongyang spies use various obfuscation techniques, including Base64 encoding, string splitting, and custom decoding routines, to hide the files’ malicious behavior.

The PowerShell script collects a ton of system information, including operating system version and architecture, system configuration, PC type, operating system installation and boot history, and a list of running processes. The attackers use this information to assess the infected environment and support follow-on attacks.

As with earlier Kimsuky campaigns, these intrusions use Git repositories for command-and-control (C2) infrastructure.

“During the analysis, Genians Security Center identified multiple public GitHub repositories operated by the threat actor,” the researchers wrote. “One repository contained not only configuration files and PowerShell scripts, but also various payloads used in subsequent attacks.”

Additionally, the months-long investigation uncovered the spies also using the Git-based C2 infrastructure for malware development and testing, stolen data management, and AI technology research.

This included setting up multiple local LLM environments using Ollama, GPT4All, and Msty on infrastructure it controlled. “Because the local approach prevents conversation data from being transmitted to external AI services, it reduces the risk of external exposure, making it a particularly attractive option for a state-sponsored threat actor,” Genians said.

The miscreants also collected a “large number” of libraries, such as LLaMaSharp and Microsoft.Extensions.AI, plus packages including OpenAI and Azure.AI.OpenAI, which call and integrate commercial AI services into their own custom applications.

“The fact that development components spanning 'local AI execution → document retrieval (RAG) → automated agents → external AI integration' were collected together strongly suggests that they were not gathered out of simple curiosity, but for the direct development of an AI-based tool designed for a specific purpose,” according to the threat hunters.

Genians uncovered logs containing speech-to-text tools, such as OpenAI’s Whisper speech recognition models, and evidence that the spies used Cursor AI to edit code and tested RAG for document-based question answering. Using RAG on stolen files can help attackers more quickly and automatically identify valuable information within large volumes of data.

While the researchers noted that they did not identify any evidence that the Norks have begun training their own models - but rather remain focused on applying AI to malware development and attack operations - the findings make a strong case for defenders needing to shift away from content-based assessment to behavior-based detection.

Assessing threats based on the quality of fake documents, such as unnatural translated language, poor formatting, and spelling errors, is no longer effective because AI is really good at producing convincing decoys.

In addition to using indicators of compromise (IoC) to detect attackers in their environments, organizations should look for anomalous behaviors following LNK execution - such as PowerShell execution, persistence establishment, and external communications - to hunt for threats.®