Today, on National Dolphin Day, researchers from Google LLC, Georgia Tech and the Wild Dolphin Project announced DolphinGemma, an artificial foundation model trained on the structure of dolphin vocalizations.

Dolphins and the myriad sounds they produce have long interested researchers as a form of communication. For decades researchers have studied the clicks, whistles and burst sound pulses of the animals as a scientific pursuit to understand what they’re saying to each other. Since 1985, the Wild Dolphin Project has conducted the longest-running study of wild Atlantic spotted dolphins in the world on their own terms.

This decades-long approach has yielded one of the largest datasets of dolphin communications in the world, including specific individuals, sounds and behaviors.

A primary focus of the WDP is to observe and analyze dolphins in their natural habitat, including their social interactions. By working underwater with them, researchers can directly link the sounds they make with specific behaviors. This allows them to directly understand how a sound – such as a pattern of clicks and whistles – can communicate unique names, activities, warnings or other signals.

To assist with this Google developed DolphinGemma, an AI model that uses audio technologies to tokenize dolphin sounds. Since the Gemma AI model is very small, with about 400 million parameters, it’s sized to fit on mobile devices and thus can be loaded onto waterproofed Pixel phones worn by WDP field researchers while underwater.

“Trained extensively on WDP’s acoustic database of wild Atlantic spotted dolphins, DolphinGemma functions as an audio-in, audio-out model, processes sequences of natural dolphin sounds to identify patterns, structure and ultimately predict the likely subsequent sounds in a sequence, much like how large language models for human language predict the next word or token in a sentence,” the researchers said in the announcement.

Using Gemma, Google is hoping to assist researchers in discovering vocabulary and understanding the unique “syntax” of patterns that may exist between the sounds they generate. This might provide insights into how they identify objects and one another during social interactions and play.

WDP is hoping to use the model to expand the research with CHAT, or Cetacean Hearing Augmentation Telemetry, an underwater computer that is designed to help establish a shared human-dolphin vocabulary. It helps facilitate communication between humans and dolphins by producing synthetic clicks, whistles and pulses – distinct from natural dolphin sounds.

Researchers use these sounds when interacting with things the dolphins might be interested in such as toys, scarves or sargassum seaweed. This way the dolphins learn to associate the synthetic sounds with the toy or seaweed and can request the object using the sounds. The CHAT can also identify which object was requested by listening for the appropriate tones.

CHAT uses a Google Pixel 6 for analysis of dolphin sounds to reduce the need for custom hardware. The Pixel phones are also smaller than a custom device, making it easier to carry them underwater, and because they’re off-the-shelf, they’re easier to maintain and replace. An upcoming generation is planned for Google Pixel 9 that will advance the research effort with its enhanced onboard deep learning model processing capabilities.

Using DolphinGemma, it’s hoped that the AI’s predictive power will help researchers analyze dolphin sounds and mimicry faster. Fine-tuning this process with different communities, or pods, of dolphins might help create a simple translatable vocabulary of objects by detecting vocalizations earlier in communication sequences so interactions are more fluid.

“By providing tools like DolphinGemma, we hope to give researchers worldwide the tools to mine their acoustic datasets, accelerate the search for patterns and collectively deepen our understanding of these intelligent marine mammals,” Google wrote.

The road to better understanding how dolphins communicate is very likely extremely long, but the advent of CHAT and DolphinGemma puts researchers one step closer to two-way communication than just listening in.

By having a better grasp of the patterns behind the sound and the ability to establish shared vocabulary with a pod of dolphins, in the future researchers may be able to get dolphins to teach them to communicate. The DolphinGemma model is trained primarily on Atlantic spotted dolphin vocalizations, but there could be a future where it can be expanded to include other species and communities of bottlenose or spinner dolphins.

With increases in fishing, trawling and shipping, the amount of noise and waste being put into the ocean continues to reduce the total habitat that they can live in. Marine noise pollution is particularly problematic for dolphins because they use sound not only to communicate but also to navigate and find food.

Google and the WDP hope that using AI will help close the gap between humans and dolphins by finally “cracking the code” of dolphin communication. This breakthrough will not only aid communication between humans and dolphins but also educate us about one of the most intelligent animal species inhabiting the oceans, which cover a significant portion of the Earth’s surface.

Image: Wild Dolphin Project