Artificial intelligence took the center stage today at Google LLC’s annual developer conference Google I/O with the unveiling of PaLM 2, the second iteration of the Pathways Language Model.

It’s a transformer-based large language model artificial intelligence developed by Google Research that powers the company’s Bard AI chatbot and other products. PaLM 2 brings a number of improvements over the original PaLM, which is a high-performance language-based AI that can scale easily and generalize across different tasks such as reasoning, conversing, creating and inferring without losing efficiency.

These enhancements include multilingual understanding, such as idioms and phrases, better logical reasoning and coding capabilities. Zoubin Ghahramani, vice president of Google DeepMind, said PaLM 2 has “significantly improved its ability to understand, generate and translate nuanced text — including idioms, poems and riddles — across a wide variety of languages” and now understands more than 100 languages.

Just like first-time learners of another language, an AI might have trouble with an idiom if it translates it literally. For example, take the German phrase, “Ich verstehe nur Bahnhof” (“I only understand train station”). Taken literally, this makes no sense. English has a very similar, equally confusing idiomatic phrase, “It’s all Greek to me.” They both mean almost the same thing: I don’t understand what you’re talking about.

The new PaLM 2’s multilingual capabilities can better understand idioms and figurative phrases across languages and recognize that these phrases could be misunderstood in translation.

According to Ghahramani, PaLM 2 can now pass advanced language proficiency exams at the “mastery” level.

As for reasoning, PaLM 2 was trained on a wide variety of scientific papers and web pages that contained mathematical expressions. This resulted in a superior reasoning engine with a greater capacity for logic. To demonstrate this, Google demonstrated that PaLM 2 could solve word puzzles such that you might find in a grade school text book concerning the order of colored cars on a shelf.

PaLM 2 has become even more versatile as a coding assistant and has been trained in a large variety of coding languages, allowing it to excel in popular languages such as Python and JavaScript. However, it can also help people in lesser-known specialized languages such as Fortran and Verilog. With its polyglot capabilities, it can also assist with documentation in multiple spoken languages.

“PaLM 2 is an extremely versatile model and it can adapt to a wide range of tasks,” said Ghahramani. Since it’s a general-purpose AI, this allowed it to be distilled into a family of models that could even be shrunk down so small it could be run on mobile devices, even when offline.

The research team has also produced specialized forms of PaLM 2 for custom uses. Med-PaLM 2, trained by health researchers with medical knowledge. It was the first LLM to perform as an “expert” test-taker level on MedQA, derived from a medical licensing test, reaching an 85% accuracy. It’s now receiving updates that will allow it to read X-rays and mammograms to one day help improve patient outcomes. In addition, there’s Sec-PaLM, a cybersecurity version of PaLM 2 developed for security use cases that helps analyze and explain the behavior of potentially malicious scripts.

Building a better Google Bard

Google announced Bard in February and rolled it out initially in the U.S. and the U.K. Since then, the company moved it onto PaLM 2, which enabled some of the more recent developments including its ability to generate and debug code in late April. According to Sissie Hsiao (pictured), vice president and general manager of Bard at Google, in the past few weeks coding has become one of the most popular things for people to do with the chatbot.

Now, Google is bringing Bard to more people by removing the waitlist and opening up access to over 180 countries, with more coming soon. The chatbot is also available in Japanese and Korean in addition to English, and is aiming to support 40 languages soon.

“As we continue to make additional improvements and introduce new features, we want to get Bard into more people’s hands so they can try it out and share their feedback with us,” Hsiao said in a blog post.

Soon, Google will make Bard more visual, and add what’s called multimodality to the interface, which means that it can include images and text. That means that when someone says something, “Tell me some interesting facts about Mt Fuji in Japan,” the chatbot will reply with pictures of Mt. Fuji. Or a user could ask for tourist spots in New Orleans or New York and a rich experience of images and text would be supplied.

People will also be able to use images, alongside their own text, when prompting the AI, in order to ask questions and make statements, and Bard will understand them and be able to gain context from the images.

That means Google Bard is slightly behind Bing Chat, Microsoft Corp.’s AI chatbot powered by OpenAI LP’s GPT-4, which launched its own visual update earlier this month. It includes an integration with OpenAI LP’s DALL-E that can produce rich generative artwork on demand using the Bing Image Creator. “To make this happen, we’re bringing the power of Google Lens right into Bard,” said Hsiao.

This will make it possible to upload an image of your cats or dogs directly into Bard and ask it to caption the image, using Google Lens, it will be able to analyze the photo, detect the animals and produce funny captions on the fly.

Additionally, Bard is receiving a few more updates including citations if it were to pull in information from the web or a block of code from another project. There will also be a dark theme to reduce strain looking at white backgrounds.

Today there will also be additional export features that allow people to push out content directly into Gmail or Docs. That way if someone asks Bard to write them a document or an email – for example if they’re drafting in the chatbot – they don’t need to copy and paste it anymore. They can simply use the export functionality to get it into Gmail or their Google Doc.

In the coming months, Google Bard will integrate with Adobe Firefly, Adobe Inc.’s own family of generative AI models for image generation. It will allow users to use their own imagination to create beautiful artwork. Additionally, they will be able to edit their AI generated artwork in Adobe Express, the online image editor.

“There’s a lot ahead for Bard — connecting tools from Google and amazing services across the web, to help you do and create anything you can imagine, through a fluid collaboration with our most capable large language models,” said Hsiao.

Photo: Robert Hof/SiliconANGLE; images: Google