Amazon Web Services Inc. is making more large language models accessible to artificial intelligence application developers through the Amazon Bedrock service, while beefing up that platform’s capabilities to optimize inference workloads and feed them with the data they need.

The latest announcements today at the company’s annual extravaganza, AWS re:Invent include the launch of the new Amazon Bedrock Marketplace. It will serve as the main portal for developers to access more than 100 of the most powerful LLMs, including some that can only be found there.

Amazon Bedrock, a fully managed service for building and scaling up generative AI applications, already provides access to some of the best-known LLMs from companies including AI21 Labs Inc., Anthropic PBC, Meta Platforms Inc., Cohere Inc., Stability AI Ltd. and Mistral AI.

With the launch of the new marketplace, available now, customers will also be able to find the newly announced Amazon Nova models, which are a new generation of foundation models announced by the company yesterday. According to AWS, the Nova models are designed to support a wide range of AI applications with industry-leading price performance.

Those aren’t the only exclusive offerings though, for Amazon Bedrock users will also get first dibs on new models from Luma AI Inc., Poolside Inc. and Stability AI.

More specifically, they’ll be able to access Luma AI’s Ray 2, which is a multimodal AI model for generative AI video creation that’s capable of creating some of the most realistic AI videos ever seen, the company promised. As for Poolside’s new Malibu and Point models, these are all about code generation, akin to GitHub Inc.’s Copilot. Stability AI’s Stable Diffusion 3.5 Large is one of the best-in-class image generation models launched in the industry thus far.

All told, customers will be able to access more than 100 popular, emerging and specialized models via the Amazon Bedrock Marketplace. Once users select the model they’re looking for, Amazon Bedrock will also suggest the most appropriate infrastructure setup for training those models and running inference, while providing simple steps they can follow to get them up and running.

Enhanced prompt engineering

In addition to more models than ever before, Amazon Bedrock users are also getting access to new techniques such as caching prompts and intelligent prompt routing, which should make it easier for developers to strike the right balance between accuracy, cost and latency.

The new “caching prompts” capability enables customers to reduce response latency and infrastructure costs by reducing repeated processing. According to the company, Bedrock does this by securely caching the most common prompts entered by users, in order to reduce costs by up to 90%, and latency by 85%.

As an example, a generative AI chat application that’s designed to answer legal questions would be able to respond much faster to the most common prompts it receives from users. In doing this, the data it refers to in order to respond to those requests will be cached within its memory, so it’s only ever processed once, and simply reused each time it receives a similar prompt. This has the effect of significantly reducing the processing costs, AWS said.

Meanwhile, the new “intelligent prompt routing” feature is designed to optimize applications for cost and response quality. Developers can configure Amazon Bedrock to automatically route prompts to different foundation models within a predetermined selection, so it will select the most appropriate model for each request or question. It aims to select the model that will give the desired response with the most accuracy and at the lowest possible cost, AWS explained. It can help to reduce overall costs by up to 30%, without sacrificing accuracy.

Expanded data access

Elsewhere, AWS is expanding the capabilities of Amazon Bedrock Knowledge Bases, which provides a way for customers to connect their models to proprietary databases in order to boost their accuracy using retrieval-augmented generation or RAG.

The company said Knowledge Bases is adding support for structured data retrieval, making it possible for AI models to query data that’s stored in traditional Structured Query Language databases. This should significantly expand their knowledge, as most generative AI applications typically only utilize unstructured data such as text, images, audio, video and so on.

It will be able to tap into structured data housed in multiple data stores, including the newly announced SageMaker Lakehouse, Amazon S3 data lakes, Amazon Redshift and others. It works by translating user prompts into SQL queries to retrieve the necessary data.

The other new capability coming to Knowledge Bases is support for GraphRAG, which enables it to create something akin to “knowledge graphs” that can map the relationships between different pieces of data stored in different locations, making it easier to retrieve.

GraphRAG makes it possible to automatically generate these graphs using Amazon Neptune, which is a specialized, fully managed graph database, without any specialist expertise, the company said.

One company already doing this is BMW Group, which is using GraphRAG to power its My AI Assistant application to help its employees and customers search for answers to their questions across its vast internal data estate, which spans hundreds of data stores.

Easier data transformation

Lastly, in a related announcement, Bedrock is getting new “data automation” capabilities that make it simple for unstructured multimodal information to be transformed into structured data, so it can be analyzed more easily.

As AWS explains, the vast majority of enterprise data is unstructured, contained in things such as documents, videos and image files. But analyzing this information is not easy, since most analytics tools only work with structured data formats.

Amazon Bedrock Data Automation is a new feature that allows Bedrock to quickly extract unstructured information from documents like PDF files and transform it into a format that these analytics tools can understand.

It should be very helpful. For instance, most banks store details of their customer’s loans on PDF files, which are tricky to analyze for insights. Traditionally, transforming these files was always a painstaking process that involved manually normalizing details such as the customer’s name and date of birth to ensure consistency.

This work can now be done much more quickly and efficiently, AWS said. Customers can just set up their predefined defaults, such as outputs based on a data schema, or scene-by-scene descriptions of video stills, and load their unstructured files into an existing, SQL-based database or data warehouse. Once there, the files can easily be analyzed.

Moreover, thanks to an integration with the revamped Knowledge Bases, Amazon Bedrock Data Automation can also be used to parse content for RAG applications, improving their accuracy and relevancy, with each response given a confidence score that helps to mitigate the risk of AI hallucinations.

Swami Sivasubramanian, vice president of AI and data at AWS, said Amazon Bedrock is enjoying rapid growth thanks to the wide selection of models and capabilities it offers. “It’s helping to tackle the biggest roadblocks developers face today, so customers can realize the full potential of generative AI,” he said.

Image: SiliconANGLE/Microsoft Designer