A new report out today from cybersecurity training services company Hoxhunt Ltd. reveals that artificial intelligence-powered phishing agents have surpassed elite human red teams in crafting effective phishing attacks, a milestone that arguably reshapes the cybersecurity threat landscape.

The report describes AI as having reached its “Skynet moment for social engineering,” a reference to “The Terminator” movie franchise, as after two years of development, AI agents developed by Hoxhunt created more effective simulated phishing campaigns than the company’s human red teams could.

The testing, which has been run three times, found that in 2023, AI was 31% less effective than humans and then in November last year, it was only 10% less effective than humans. Forward to March and AI was found to be 24% more effective than humans.

The research used Hoxhunt’s proprietary AI spear phishing agent, internally known as JKR — short for Joker — which was tested across millions of enterprise users. The AI agent was designed to create and refine highly targeted phishing emails using user-specific context such as role and location. Over time, the agent evolved using an internal process called “Evolves,” which continuously fine-tuned its prompts and model output, leading to significant performance gains in just a few months.

The current testing differed from earlier tests in 2023 that relied on simple, single-prompt responses from large language models with the use of sophisticated AI agents that are capable of both generating novel phishing content and enhancing human-created messages. The advanced methodology allowed the AI to craft attacks that were far more convincing and adaptive, effectively matching or exceeding human creativity and intuition in social engineering.

While the testing was all done in-house by Hoxhunt’s researchers using their own tools, it demonstrates that the underlying technology is rapidly advancing. Hoxhunt suggests that once AI spear phishing agents are integrated into phishing-as-a-service offerings, the baseline quality of mass phishing attacks will rise to levels previously associated only with highly targeted spear phishing operations.

As the technology advances, the trends are only going to get worse. Today, AI-written phishing emails that bypass email filters still make up less than 5% of total volume, but since ChatGPT’s debut in 2022, the number has grown by 4,151%. Hoxhunt also found that there has been a 49% increase in phishing attacks that evade email filters in that time.

The problem is clear: As AI-generated attacks become easier to create and more lucrative, they’ll see broader adoption by cybercriminals.

All hope is not lost, however, with behavior-based phishing training remaining effective against both human and AI-generated threats. Adaptive training platforms that simulate real threats and personalize user education are also proving more resilient, according to Hoxhunt’s findings.

“As AI technology continues to evolve, the ability to craft more sophisticated phishing attacks on demand will only increase, making AI an essential tool in both offensive and defensive cybersecurity strategies,” wrote Hoxhunt co-founder and Chief Technology Officer Pyry Åvist.

Image: SiliconANGLE/Reve