Small language models are revolutionizing the field of artificial intelligence, especially within the public sector.

As generative AI continues to advance, key discussions are now centered on enhancing data security and privacy, which are crucial for enterprises handling sensitive information. This shift toward small, efficient language models tailored for specific use cases is gaining momentum, offering businesses cost-effective and secure solutions for AI deployment.

“What we do is we build a small language model that can be deployed privately in customer’s on-premise or private cloud environment,” said Kasey Roh (pictured), head of U.S. business at Upstage Co. Ltd. “Enterprises have a lot of confidential customer data or public data. They can’t take it outside of their containerized database and want to train the model using that specific data, and also run and operate and manage the data model within their own cloud environment. With that end-to-end data pipeline along with the model is what we offer at Upstage.”

Roh spoke with theCUBE Research’s John Furrier at the AWS Summit Washington, DC, during an exclusive interview on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the benefits of small language models for data privacy and cost efficiency, the distinction between large and small language models, and Upstage’s strategic partnerships and initiatives.

Small language models: A cost-effective solution for data privacy

Upstage, an enterprise AI startup, specializes in building small language models that can be deployed privately, either on-premises or within private cloud environments. The primary focus of Upstage is to address the critical issues of data security and privacy, which are at the forefront of discussions in the public sector, according to Roh.

“What we offer is having a small size model that is very good at one use case, one specific use case,” she said. “We have a base model that is reasonably strong in terms of the performance and accuracy but can be further improved once it’s customized when further trained with the customer’s private data.“

The main distinction between large language models and SLMs is that LLMs are typically used for broader applications and often come with significant cost and scalability challenges. In contrast, SMLs, such as those developed by Upstage, are designed for specific use cases and can be deployed more cost-effectively, according to Roh.

“What we typically do is [add] additional layers of safety or the assurance for the accuracy,” she said. “That comes down to the RAG application or we have a groundness checklist, which means once the model spits out the answer, the answer goes back to the sourcing data, the underlying data to double-check the accuracy of the data.”

Strategic partnerships and evolving startup landscape in AI

Upstage’s strategic partnerships play a crucial role in the company’s operations. The company collaborates closely with Amazon Web Services Inc., using its environment for training models and hosting them on the AWS Marketplace and SageMaker. This partnership enables Upstage to package its model offerings with AWS cloud services, facilitating seamless integration and enhancing model training and deployment processes.

“AWS has been a phenomenal partner for Upstage,” Roh said. “We are certainly training our model in the AWS environment, but our models are also hosted on AWS Marketplace and our models are available on SageMaker. When we meet the customers who are either already in AWS, their data is already on AWS or they are considering being in AWS environment, we are able to package our model offering with the AWS cloud environment to supercharge the model training and the continuous serving of the model.”

The startup environment in the AI industry is dynamic and rapidly changing. While the initial focus was on building foundational models, the current trend is shifting toward developing application-specific solutions, Roh noted. Entrepreneurs and developers are increasingly leveraging different sets of models to address specific business problems, resulting in a surge of vertical solutions across various sectors, including legal, healthcare and public services.

“We are seeing a lot of vertical solutions, leveraging certain problems the lawyers are facing,” she said. “In this DC Summit, we’re seeing a lot of solutions that are trying to tackle the public use cases, like how can federal government use the foundation models? So, I think that the next opportunity is really at the application layer, leveraging the foundation model.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE Research’s coverage of the AWS Summit Washington, DC:

Photo: SiliconANGLE