A new report out today from cloud-native application security firm Sysdig Inc. details one of the first instances of a large language model being weaponized in an active malware campaign.

Discovered by Sysdig’s Threat Research Team, the malware campaign involved exploiting misconfigured instances of Open WebUI, a widely used self-hosted artificial intelligence interface, to deploy malicious, AI-generated payloads targeting both Linux and Windows systems.

The attack began when a training system using Open WebUI deployed by one of Sysdig’s customers was mistakenly exposed to the internet with administrative privileges and no authentication. The exposure to the internet allowed anyone to execute commands on the system, dangerous mistake attackers are well aware of and actively scanning for.

Open WebUI, which has more than 95,000 stars on GitHub, allows extensible enhancements for large LLMs via custom Python scripts. The attacker exploited the feature by uploading a malicious, obfuscated Python script through Open WebUI’s plugin system. The system’s internet exposure and lack of safeguards provided an easy entry point for the attacker to execute commands and deploy further malicious payloads.

The uploaded Python script was obfuscated using PyObfuscator and also contained a distinctive style indicative of AI-generated code. The script, which underwent multiple decoding layers, downloaded and executed crypto miners targeting Monero and Ravencoin networks, while establishing persistence via a systemd service masquerading as “ptorch_updater.”

Notably, the use of inline format string variables, a common feature in AI-generated code, was prevalent throughout the malicious script. Sysdig’s researchers confirmed that parts of the code were likely AI-generated or heavily AI-assisted, a trend that could signify a shift towards the rapid development of malware using generative AI tools.

What was targeted by the malware depended on the system. On Linux, the payload focused on crypto jacking, deploying miners like T-Rex and XMRig while employing defense evasion tools. On Windows, the malware installed a Java Development Kit to run a malicious file that included components designed for credential theft, sandbox evasion and hardware discovery.

The good news, as much as there can be in malware cases, Sysdig’s runtime threat detection was able to identify the threat in real time. Using a combination of YARA rules, behavioral detections and threat intelligence, Sysdig detected the suspicious activity, including unauthorized code compilation, domain lookups, and the use of known miner communication protocols.

The malware attack and campaign highlight emerging AI threats when it comes to malware and other forms of cyberattack and in this case, on both ends — malware written by AI that then exploits LLMs on the end of the attack. The Sysdig researchers also noted that the case emphasizes the need for runtime security with multilayer threat detections.

Image: SiliconANGLE/Reve