The race to serve AI inference faster and cheaper is exposing the hard limits of conventional chip architecture. As demand for real-time AI responses accelerates, the industry’s standard response — stacking more high-bandwidth memory onto power-hungry silicon — is running into a wall, and logarithmic math may be the foundational rethink that breaks through it.

Fresh out of stealth and with its first chip now in production at Taiwan Semiconductor Manufacturing Co., Ltd., Tensordyne Inc. is positioning itself to challenge the AI inference market by rearchitecting the math inside the silicon — not just the chip itself — according to Gilles Backhus (pictured), co-founder of Tensordyne.

“Our logarithmic math — it’s completely under the hood,” Backhus said. “From a user point of view, from an SDK point of view, you don’t even notice it. It just looks like normal floating-point math. It’s just that the engine under the hood is more efficient.”

Backhus spoke with theCUBE’s John Furrier at the RAISE Summit, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how Tensordyne’s cross-continental co-design model, logarithmic math innovation and Juniper Networks-derived rack architecture are positioning the company as a direct challenger to Nvidia Corp. in the AI inference infrastructure market. (* Disclosure below.)

Logarithmic math eliminates multiplier circuits to shrink power and footprint

The key insight behind Tensordyne’s approach is eliminating the most expensive circuits on an AI chip. Traditional silicon devotes enormous transistor area to multiplier circuits needed for floating-point arithmetic, but multiplications can be converted to additions in logarithmic space — and adders are dramatically smaller and more power-efficient than multipliers. Tensordyne has filed foundational patents on its proprietary “Pareto” logarithmic number system and solved the hard part: accurately and cheaply converting back to linear representation for the additions that must still occur, Backhus noted.

“In LogMath, instead of representing numbers as an exponent and a mantissa like in floating point, you now only have an exponent, but it’s a fractional exponent,” he said. “If you want to multiply two numbers, suddenly what you do is you only add the two exponents of the two numbers. That transformation from a logarithmic representation to a normal one is normally so expensive, but we found a way to do this extremely accurately and still very cheap.”

The result is a 72-chip inference pod that fits in just 13 rack units, draws only 30 kilowatts — compared with 150 kilowatts for a comparable Nvidia system — and uses all-copper, single-hop chip-to-chip interconnects at roughly one microsecond of latency, about 10 times lower than competing architectures. That density means four pods fit in a single standard rack, enabling customers to serve the largest frontier models at more than 1,000 tokens per second per user without requiring multiple racks or a secondary high-speed networking provider, Backhus said.

“You can have basically a state-of-the-art data center [in a single rack],” he said. “At least we can serve the largest models at the highest speeds in a single rack. You don’t need multiple racks. So your threshold, your entry threshold now to be part of the most premium tier of serving tokens is now a single rack.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the RAISE Summit:

(* Disclosure: TheCUBE is a paid media partner for the RAISE Summit event. Neither Solidigm, the headline sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE