Generative artificial intelligence startup Hume AI Inc. said today it has closed on a $50 million funding round after creating an AI chatbot that brings realism to the next level.

The Series B round was led by EQT Ventures and saw participation from Union Square Ventures, Nat Friedman & Daniel Gross, Metaplanet, Northwell Holdings, Comcast Ventures and LG Technology Ventures.

Hume AI’s flagship product is an emotionally intelligent voice interface that can measure human speakers’ emotional expression so it can better understand the nuances of what they’re saying. Available now in beta test, Hume AI’s Empathic Voice Interface is a conversational AI that sets itself apart from the likes of ChatGPT by comprehending the tone of voice used by the people it’s talking to, adding depth to each interaction and tailoring its responses accordingly.

EVI is said to be trained on data from millions of human interactions. It can understand when the user has finished speaking and generate an appropriate vocal response almost instantaneously.

The startup’s founder, Chief Scientist and Chief Executive Alan Cowen (pictured, center) helped to pioneer the concept of semantic space theory during his time as an AI researcher at Google LLC. Semantic space theory is a computational approach to understanding emotional expression and experience.

It’s this underlying technology that helps the startup’s EVI to get a better grip on the nuances of human voice. The startup said its goal is to enable more engaging and realistic voice-first generative AI experiences that accurately emulate the natural speech patterns of human conversation.

The EVI was built using a kind of multimodal generative AI that combines standard large language model capabilities with expression measuring techniques. The company calls this novel architecture an “empathic large language model” or eLLM, and says this is what allows EVI to adjust the words it uses and the tone of its voice, based on the context and emotional responses of human speakers.

It’s clever, with the ability to accurately detect when the speaker is ending their conversational turn, so it can start responding almost immediately, with latency of less than 700 milliseconds. It’s sensitive too, and it will stop speaking if the user interrupts it. It all adds up to a much more fluid, humanlike conversational interaction.

Unlike with many other generative AI chatbots, which are known for the slow and somewhat mechanical nature of their conversations, chatting with Hume AI’s EVI genuinely feels like talking with a real human being. The startup is inviting people to check it out here, and users can jump right in with no need to sign up.

Hume AI intends to make its application programming interface available in beta next month, enabling developers to integrate it with any app and offer a more immersive and empathetic chat experience. The API is said to include not just the eLLM, but also tools for measuring the human emotional expression that is necessary to facilitate its realistic chats.

Besides its empathic conversational capabilities, EVI supports fast and reliable transcription and text-to-speech functionality, meaning it can adapt to a wide range of scenarios. Developers will be able to enhance it even further by integrating it with other LLMs.

EVI understands the user’s tone of voice, which adds meaning to every word, and uses it to guide its own language and speech. Developers can use our API as a voice interface for any application. Try it here: https://t.co/5yexH0wW4W

— Hume (@hume_ai) March 27, 2024

In a post on LinkedIn announcing today’s funding round, Cowan said he believes voice interfaces will ultimately become the default way most people interact with AI.

“Speech is four times faster than typing; frees up the eyes and hands; and carries more information in its tune, rhythm, and timbre,” he said. “That’s why we built the first AI with emotional intelligence to understand the voice beyond words. Based on your voice, it can better predict when to speak, what to say, and how to say it.”

EQT Ventures Partner Ted Persson said the startup’s empathic models are the crucial missing ingredient in the AI industry. “We believe that Hume is building the foundational technology needed to create AI that truly understands our wants and needs, and are particularly excited by its plan to deploy it as a universal interface,” he said.

The startup has already seen some traction, rolling out a beta version of its EVI last September to a waitlist of more than 2,000 companies and research organizations, with a primary focus being on healthcare applications. To that end, it has built research partnerships with labs at Mt. Sinai, Harvard Medical School and Boston University Medical Center, where it has explored ways in which the understanding of a patient’s nuanced vocal and facial expressions can help to improve outcomes. Some of its existing applications include standardized patient screening, triage, targeted diagnosis and treatment for mental health conditions.

Holger Mueller of Constellation Research Inc. said it’s notable that most other generative AI companies have been more focused on ensuring their chatbots get their facts right, so as to avoid controversy. But although that’s important, if generative AI is going to change the future of work it must also get the human lymphatic site right as well. “It’s good to see this level of innovation with Hume AI tackling the issue,” he added. “A key question will be whether or not Hume AI’s foundational model architecture can merge with other AI models.”

“What sets Hume AI apart is the scientific rigor and unprecedented data quality underpinning their technologies,” said Union Square Ventures Managing Partner Andy Weissman. “Hume AI’s toolkit supports an exceptionally wide range of applications, from customer service to improving the accuracy of medical diagnoses and patient care, as Hume AI’s collaborations with SoftBank, Lawyer.com and researchers at Harvard and Mt. Sinai have demonstrated.”

Although it seems to be at the cutting edge of generative AI chat experiences, Hume AI is likely to face some stiff competition as the technology evolves. For instance, ChatGPT creator OpenAI is said to be working on a “Voice Engine” that will incorporate voice and speech recognition capabilities, as well as speech and voice generation features that will enable it to respond to natural language prompts in a more realistic way.

Photo: Hume AI