Intel Corp.’s venture capital arm has co-led a $125 million funding round for chip startup Untether AI Inc., whose artificial intelligence processor uses an emerging approach dubbed at-memory computation to run machine learning algorithms. 

The other lead investor in the round, which was announced this morning, is Tracker Capital. The Canada Pension Plan Investment Board and Radical Ventures participated as well.

Toronto-based Untether AI has developed a processor called the runAI200 that is optimized for AI inference. Inference is the term for running AI algorithms in production on live data after they exit the development and training phase. According to the startup, a single-server accelerator card with four runAI200 units can manage about 2 quadrillion operations per second while achieving “industry-leading” power efficiency. 

The key to the runAI200 processor’s speed is a unique architecture that combines memory and computing elements in the same chip. Untether AI is calling this architectural design approach at-memory computation.

In a traditional server, the compute and memory components are separate from one another. The ones and zeros that make up an AI model are kept in memory. When they’re needed, they travel from a server’s memory to its processor, which carries out computations and then sends new data back the other way around for storage. This constant movement of information slows down computations because the processor has to wait for data to arrive from the memory component before it can start performing calculations. 

According to Untether AI, data movement accounts for as much as 90% of the electricity consumed by AI applications.

The startup’s runAI200 processor decreases data movement, and thereby reduces the delays involved in processing the data, by combining the usually separate memory and compute components into a single chip. A runAI200 processor is made up of circuits dubbed memory banks by Untether AI. Each memory bank combines 512 miniature compute modules with 385 kilobytes of memory. Because the memory and compute elements are in the same place, data takes less time to travel between them than if they were implemented as separate components.

Untether AI sells the runAI200 as part of an accelerator card called TsunAImi that companies can attach to their servers. With its ability to perform up to 2 quadrillion operations per second, the TsunAImi can run the standard version of the BERT machine learning model four times faster than any other product, Untether AI claims. BERT is an industry-standard algorithm for natural language processing that is extensively used in AI projects.

In the power efficiency department, Untether AI says that its at-memory computation approach reduces the amount of electricity needed for data transfers between the memory and compute elements by up to six times. The startup claims this allows the TsunAImi to perform about 8 trillion processing operations per watt of electricity consumed.

Untether AI believes its silicon could be applied in a wide range of areas. Autonomous driving is one use case the startup is targeting. In theory, the power efficiency of the startup’s chip could make it easier for automakers to run AI-powered autonomous driving software on electric vehicles while optimizing their battery life. Financial services and retail are two other segments where Untether AI sees potential sales opportunities.

The startup will use the new $125 million provided by Intel and its other investors to “accelerate and expand its customer engagements” across multiple markets. Additionally, Untether AI will use a portion of the funds for product development, with a particular focus on enhancing its software portfolio. The startup offers multiple software tools alongside its chips, including an application that automatically optimizes AI algorithms developed with frameworks such as TensorFlow to run on the runAI200.

What makes Intel’s investment in Untether AI particularly notable is that the chip giant offers its own processor for AI inference. It acquired the processor via the $2 billion acquisition of semiconductor startup Habana Labs Ltd. in 2019. Similarly to Untether AI, Habana Labs received funding from Intel via its venture capital arm. 

Elsewhere in the chip market, other startups besides Untether AI are also experimenting with placing large amounts of memory on their chips to speed up inference. One of them is Graphcore Inc., which raised a $222 million round late last year. Each Graphcore chip has 900 megabytes of memory that can be used to store an AI model closer to the silicon responsible for running it and thus speed up computations.

Photo: Unsplash