Artificial intelligence governance startup Endor Labs Inc. today announced the availability of a useful new tool that will allow companies to search and discover which AI models are being used in their business applications, so they can create and enforce security policies to safeguard their private data.

Called AI Model Discovery, the tool is aimed at addressing a trio of key use cases. For starters, it gives application security teams a way to check which open-source AI models have been deployed within their applications. Second, it allows them to evaluate these models for any risks. Third, it provides a way for teams to enforce organization-wide security policies relating to AI models.

Besides identifying which AI models are being used in each application, it also provides alerts for policy violations, and can be used to block the highest-risk models from being used in production.

The new capability is being integrated with Endor Labs’ Dependency Lifecycle Management Platform, which uses AI to analyze the open-source dependencies used within a company’s applications, so they can be monitored and maintained at large scale. It helps developers identify the safest dependencies by examining key metrics across security, quality, popularity and activity, so they can assess the security and operational risks of using those dependencies. It also helps prioritize fixing the most critical vulnerabilities.

Endor Labs’ co-founder and Chief Executive Varun Badhwar said there’s a significant gap in the ability of companies to use AI models safely, because traditional software composition analysis tools are focused on identifying open-source packages rather than AI algorithms. “This means they usually can’t identify the risks from local AI models integrated into an application,” he said.

The company said the AI Model Discovery builds on its existing Endor Scores for AI Models capability, which uses 50 metrics to evaluate the safety of every AI model available on the Hugging Face platform, based on their popularity, security, quality and activity.

Endor Labs says the ability to check how safe AI models are is essential, because most companies don’t have the time or the expertise to create their own models. Instead, almost everyone relies on open-source models, which they adapt to fit specific use cases and requirements. These models thus become application dependencies in their own right, similar to other open-source components. But they cannot be analyzed properly by existing vulnerability scanners.

Katie Norton, an analyst with International Data Corp., said many enterprises have overlooked the need to secure the open-source AI components they use, simply because the tools to do this were not easily available before.

“Our research finds that 60% of organizations choose open-source models over commercial ones in their most important generative AI initiatives, so finding and securing those components is critical for any dependency management program,” she said. “Vendors like Endor Labs are addressing an urgent need by integrating AI component security directly into software composition analysis workflows, while providing meaningful remediation capabilities that don’t overwhelm developers.”

By scanning each open-source model to understand the risk, teams can then build the necessary guardrails to ensure each one can be deployed safely within their applications. Developer teams will gain the same kind of visibility and control over open-source AI as they have over other software components, the company said.

AI Model Discovery is available starting today for all existing customers.

Image: SiliconANGLE/Microsoft Designer