A new report released today by researchers at cloud-native security company Aqua Security Software Ltd. warns of a new attack targeting Apache Hadoop and Flink applications.

The attack is described as “particularly intriguing” thanks to the attacker’s use of packers and rootkits to conceal their malware. It was discovered by the researchers over the last few weeks in their cloud honeypots, networks set up to attract and monitor cyberattackers to understand their techniques and improve defenses.

The attackers were observed exploiting a misconfiguration in the ResourceManager of Hadoop YARN. That component of Hadoop provides a platform for managing computing resources in clusters.

With the exploit, the attackers target Hadoop YARN to gain unauthenticated access to create and run applications. The misconfiguration can be exploited by an unauthenticated, remote attacker through a specially designed HTTP request, potentially leading to the execution of arbitrary code, depending on the user’s privileges on the node where the code is executed.

The same honeypots also detected attacks on Apache Flink, an open-source, unified stream-processing and batch-processing framework developed by the Apache Software Foundation. Though that’s a different vulnerability, attackers were also observed gaining access in a similar manner to the attacks on Hadoop.

The attacks follow a sophisticated flow, starting with gaining initial access and then deploying primary and secondary payloads. Attackers take steps to evade defenses and align their techniques with various tactics in the MITRE ATT&CK framework, indicating a high level of sophistication.

The development of the attack methodology used raises concerns regarding the security of big data. In the words of the researchers, it presents a “significant challenge to traditional security defenses.”

The researchers advise that big data operators implement agent-based runtime solutions, which are said to be a good solution for detecting suspicious or malicious activities, such as crypto miners, rootkits and obfuscated binaries. The solutions are also adept at identifying and mitigating container drift, a deviation or changes that occur in a container’s runtime environment compared to its original, intended state, often leading to security vulnerabilities or operational inconsistencies.

Image: Hadoop