Hailo Technologies Ltd., a startup that specialized in making computer chips for artificial intelligence workloads at the edge, said today it has raised another bumper funding round.

It has just closed on a $120 million Series C extension led by current and new investors such as the Zisapel family, Gil Agmon, Delek Motors, Alfred Akirov, DCLBA, Vasuki, OurCrowd, Talcar, Comasco, Automotive Equipment and Poalim Equity. The latest round brings its total amount raised to date to more than $340 million.

Alongside the round, the startup announced the newest addition to its chip lineup, launching the Hailo-10 high-performance generative AI accelerator, which it says will usher in a new generation of lower latency AI models that can run locally at the edge, without connecting to cloud-based servers.

Hailo was founded in 2017 and first made headlines in 2019 when it launched a customized processor for running deep learning workloads at the edge of the network. That chip, called the Hailo-8 Deep Learning chip, was designed to power intelligent, autonomous vehicles, smart cameras, drones and augmented reality and virtual reality platforms. It gave them the ability to run AI locally, meaning they can process data and deliver results much faster than they would if they had to connect to a distant cloud-based server.

The startup, which also provides software tools for developers to build applications that are customized to run on its hardware, has since built on that initial offering, launching a range of more powerful, focused chips for different AI workloads. One of its more recent additions is the Hailo-15 processor family, which launched in March 2023. The Hailo-15 chip is designed for vision AI, especially smart cameras that can analyze the footage they record locally.

Most recently, it announced an upgraded Hailo-8 chip lineup, including a new, high-end Hailo Century PCIe model that adds a Peripheral Component Interconnect Express card to support more advanced models at the edge, plus a low-end offering called Hailo-8L for entry-level uses.

Bringing generative AI to the edge

The newest addition to Hailo’s chip family, the Hailo-10, is the company’s first chipset designed specifically for generative AI workloads that run at the network edge. It provides the advantage of continuous access to generative AI services, eliminating the problem of low latency that impacts many existing large language models, while also promoting greater privacy by eliminating the need to send data to the cloud. It also helps keep personalized information anonymized, the company said, while enhancing sustainability with its more efficient on-board processing capabilities.

The startup said unlocking generative AI on edge devices such as smart vehicles, personal computers, smartphones and robots will enable users to truly “own their generative AI experiences” and make them an integral part of their lives.

The Hailo-10 architecture has been designed to deliver maximum performance with minimal power, said Hailo co-founder and Chief Executive Orr Danon. “As gen AI on the edge becomes immersive, the focus turns to handling large LLMs in the smallest possible power envelope — essentially less than five watts,” he said.

The startup said the Hailo-10 can run the open-source Llama2 7B model with up to 10 tokens per second using less than five watts of power. In addition, it’s rated at under five seconds per image when using Stability AI Ltd.’s popular Stable Diffusion 2.1 image generation model with the same power envelope.

According to Hailo, the Hailo-10 can deliver a maximum of 40 tera operations per second, which represents a new standard for edge-based AI accelerators. It’s faster and more energy-efficient than integrated neural processing units, delivering at least twice the performance of Intel Corp’s Core Ultra NPU, according to published benchmark tests.

The startup said it sees the Hailo-10 generative AI accelerator being deployed in PCs and car infotainment systems, replacing the existing central processing units that are widely used in such systems. Those CPUs are not capable of supporting chatbots, personal assistants, copilots and similar AI applications, which means those kinds of applications have always had to connect to the cloud, leading to lower latency and slow performance. Hailo-10 promises to change that, with its high performance providing almost instantaneous responses, the startup said.

The company will begin shipping samples of the Hailo-10 generative AI accelerator in the second quarter, so we can expect to see them appear in the real world by the end of the year.

“Whether users employ generative Ai to automate real-time translation or summarization services, generate software code, or images and videos from text prompts, Hailo-10 lets them do it directly on their PCs or other edge systems, without straining the CPU or draining the battery,” Danon said.

Danon said the newly raised money will help the startup capitalize on the numerous opportunities in its pipeline and set the stage for its long-term growth. Its mission is to accelerate both classic and generative AI services at the edge, expanding the technology’s reach to more users.

Image: Hailo