A new report out today from application security firm F5 Inc. reveals that bots now generate more than half of all web content page requests, with generative artificial intelligence providers driving a sharp increase in automated traffic.

The findings come from the F5 2025 Advanced Persistent Bots Report, based on an analysis of 207 billion web and application programming interface transactions between November 2023 and September 2024.

The report found that half of content-related web traffic came from automated sources, particularly large language model scrapers used by companies like OpenAI, Anthropic PBC and Perplexity AI Inc. Beyond LLM and AI bots, 22.3% of bot traffic was found to be search requests on the web and 21.5% were add-to-cart transactions.

The AI bots are noted in the report as not only being persistent but also often bypassing countermeasures to extract valuable data such as product listings and pricing.

Of the transactions monitored, 21.22 billion transactions — 10.2% — came from a variety of automated sources, some of them benign. But 10 billion, or 4.8%, consisted of malicious bot traffic.

“For years, bot traffic has primarily been targeted at search flows, as well as aspects of the user journey where someone signs up or logs in to use a service, adds an item to their basket, checks out, or seeks to change their password,” said F5 Labs Director David Warburton. “The huge upsurge in content scraping, undoubtedly associated with the explosion of generative AI and LLMs, underlines how dynamic bot traffic is and the need for organizations to be constantly on watch for changes in attack patterns.”

Other findings in the report include wide variation in bot traffic patterns across industries. The most targeted sectors were hospitality, with came in at 44.6%, healthcare at 32.6% and e-commerce on 22.7%. Mobile traffic was dominated by attacks on entertainment — 23% — well ahead of other industries.

Credential stuffing attacks are also noted in the report to continue to be a major concern, particularly in the technology sector, where 33.5% of login attempts were classified as account takeovers.

The report also highlights differences in attack sophistication across industries. While healthcare was primarily targeted by basic bots, sectors like general retail, banking and airlines faced a higher proportion of advanced and persistent automated threats.

While the figures are negative, there is some good news in the report as well, with the majority of industries tracked experiencing a decline in automated activity compared to 2023, suggesting that bot controls are having the desired effect.

Image: SiliconANGLE/Reve