Snowflake Inc. is using its Snowday event this week to roll out a huge batch of new and enhanced services, including many related to generative artificial intelligence.

In a bid to simplify the development of AI models, the company is unveiling a private preview of Snowflake Cortex, a managed service that’s said to make it easier for organizations to discover, analyze and build AI applications in the Snowflake Data Cloud. Cortex provides access to serverless functions that include Meta Platforms Inc.’s Llama 2 large language model as well as task-specific models and advanced vector search functionality. Snowflake said the combination can be used to build contextualized LLM-powered applications in minutes.

“We want to make it easy for customers to use AI and put generative AI in the hands of every user,” said Sridhar Ramaswamy, a Snowflake senior vice president.

Cortex doesn’t require users to have specialized AI expertise, the company said. Its serverless functions can be invoked through SQL or Python code, and specialized functions can be used for tasks like sentiment analysis, text summarization, translation and conversational interactions with data.

Two new serverless function sets are available in private preview that can be invoked with a function call in SQL or Python code. Specialized Functions are specific to tasks like detecting sentiment, extracting an answer, summarizing text and translations. General-purpose functions are conversational in nature and include text-to-SQL capability so users can “chat” with their data.

Better SQL

“We have been working on a model that is much better at generating SQL than any of the open-source models, and we are demonstrating that through Cortex,” Ramaswamy said. “It looks like an [application program interface], but it runs with Snowflake and data never leaves or is intermingled for training.”

Streamlit in Snowflake helps accelerate custom LLM-based application development by turning data, AI models and analytic functions into interactive applications written in Python.

In conjunction with the Cortex rollout, the company is also providing private previews of three LLM-powered applications built with the service.

Snowflake Copilot is an LLM-based assistant that lets users ask questions of data in plain text, write and refine SQL queries and filter results. Universal Search can search across a customer’s Snowflake account, including databases, views and Iceberg tables as well as applications in the company’s marketplace. Document AI can extract content like invoice amounts or contractual terms from documents and fine-tune results using a visual interface and natural language.

For users who want full LLM application customization, Snowpark Container Services simplifies the secure deployment and management of containerized workloads. It gives developers the ability to run third-party applications, including those from commercial LLMs and vector databases, entirely within their Snowflake account.

Enhanced developer tools

For developers, Snowflake is enhancing Python support in its Snowpark platform with support for workloads running in software containers and expanded DevOps capabilities. Snowpark allows software engineers to deploy custom code on top of Snowflake’s cloud data warehouse. Three other new Snowpark features are aimed at machine learning model development.

Snowflake Notebooks provides an interactive, cell-based programming environment for Python and SQL users. A new Snowpark machine learning API lets developers and data scientists scale feature engineering and simplify model training for faster and more intuitive model development in Snowflake.

The Snowpark Model Registry builds on a native Snowflake model entity to enable scalable and secure deployment and management of models in Snowflake, including expanded support for deep learning models and open-source large LLMs from Hugging Face Inc.

“A large percentage of models don’t get into production because they are living in a vacuum,” said Jeff Hollan, director of product management. “With the Model Registry, they are now a first-class source that you can register with the right metadata so anybody can discover and deploy them.”

Also for developers, Snowflake announced that the Native App Framework it introduced more than 16 months ago will soon be available in the Amazon Web Services Inc. cloud and in a public preview on Microsoft Corp.’s Azure cloud. The framework is essentially an “app store” for enterprises. “It provides building blocks for every organization to perform all the steps from building the app to deploying it privately to exposing and monetizing it,” said Prasanna Krishnan, senior director of product management, collaboration and Snowflake Marketplace.

The new Snowpark Container Services lets developers run any component of their application — including ML training, LLMs and application programming interfaces — without having to move data or manage container-based infrastructure. It will be in public preview soon in select AWS regions.

Apache Iceberg support

There’s also good news for fans of Apache Iceberg, the open-source table format for large-scale data lakes. Snowflake said it’s advancing support for Iceberg Tables to enable users to unite all their data in one place and access it data from other engines. It will be in public preview soon.

New governance capabilities are being made available in Snowflake Horizon, a governance platform with a unified set of compliance, security, privacy, interoperability and access capabilities. The platform now covers a variety of additional authorization and certifications, including the U.K.’s Cyber Essentials Plus, FBI’s Criminal Justice Information Services Security Policy and the Internal Revenue Service’s Publication 1075 Tax Information Security Guidelines.

A new Data Quality Monitoring service simplifies the measurement and recording of data quality metrics for reporting, alerting and debugging. Packaged and custom metrics are available. A Data Lineage user interface provides a high-level visualization of the upstream and downstream lineage of objects. Custom Classifiers (in private preview), international classification (generally available) and Snowflake’s new user interface-based classification workflow (public preview) allow users to define sensitive data and identify it across their organization.

Snowflake also launched a Powered by Snowflake Funding Program that will invest up to $100 million in early-stage startups building Snowflake Native Applications. The program is being launched in conjunction with several high-profile venture capital firms and AWS, which is providing $1 million in Snowflake credits on AWS to startups building Snowflake Native Applications.

Image: Pixabay