Application security solution provider White Source Ltd., also known as Mend.io, today launched System Prompt Hardening, a dedicated capability designed to detect issues within the hidden instructions given to large language models before they run to strengthen logic and reduce risk.

A system prompt is a foundational, hidden set of instructions given to an AI model before it begins a conversation. It defines its interaction parameters, role, tone, behavior and constraints. It might be something as simple as: “You are a friendly, polite assistant,” or as a multi-line complex set of rules defining edge cases, boundaries and context for behavior.

“System prompts are the behavioral blueprint for AI applications, but security standards haven’t kept pace with their growing importance,” said Rami Sass, general manager of Mend.

According to Gartner Inc., 32% of organizations reported experiencing an attack on AI applications that used the application prompt in the past year.

In fact, it is particularly common to use prompt injection — malicious hidden instructions used to override original programming — to get AI applications to reveal the system prompt because knowing the default instructions can be used to manipulate them.

Mend.io said its new system prompt hardening capability helps move security teams beyond ad hoc testing and manual red teaming to test LLM responses to attacks in a standardized framework for managing security.

“While security and development teams have established frameworks like Common Weakness Enumeration and Common Weakness Scoring System to evaluate software risk, we are now introducing System Prompt Hardening and AI Weakness Enumeration as the first formal methods to assess and enhance these instructions,” Sass added.

Due to their central authority in interaction with outside sources, and a clear opportunity for attackers to circumvent and use as vectors for manipulation, Mend.io said its new system treats system prompts as high risk.

The new solution uses automated detection and labelling to provide actionable context for security and development teams to understand how to classify issues that could be exploited by attackers. The system provides 1-100 scoring on a severity scale that quantifies risk categories of prompt vulnerabilities for triage and remediation priority based on impact.

Going beyond simple detection, the platform also recommends refinements to prompt logic — otherwise known in the industry as “hardening” — to neutralize prompt injection threats before they can affect production applications.

“As organizations accelerate AI adoption across their application portfolios, system prompts increasingly represent a material attack surface for injection and manipulation risks,” said Katie Norton, research manager for DevSecOps and software supply chain Security at International Data Corporation. “Treating system prompts as governed artifacts rather than ad hoc instructions reflects a maturing approach to AI security.”

The company said this new capability is available in Mend AI Core and Mend AI Premium, the company’s core platform automation security for AI applications for discovering, assessing risk and repairing AI components with enforced policy and compliance.

Image: SiliconANGLE/Microsoft Designer