

New research out today from Knostic Inc., a startup that provides need-to-know-based access controls for large language models, details a new category of vulnerabilities in LLMs that can be used by attackers to bypass guardrails and extract sensitive information.
The vulnerabilities, called #noRAGrets, consist of two specific vulnerabilities that can entirely bypass model guardrails through a “race condition-like” attack, affecting artificial intelligence chatbots such as ChatGPT and Microsoft Copilot for Microsoft 365. A race condition attack in AI exploits the timing of operations within a system to manipulate or bypass safeguards, causing unintended or unauthorized behaviors.
The new exploitation methods are said by Knostic’s researchers to take the “language tricks” of jailbreaking attacks to a new level by using timing techniques to allow attacks to bypass guardrails entirely and manipulate internal LLM application activity.
“LLM applications and agentic systems are more than just the model and the prompt,” said co-founder and Chief Executive Gadi Evron. “They have multiple components besides the model, from the user interface to the guardrails, all of which can be attacked on their own or by gaming the interplay between them.”
One example: By exploiting one of the vulnerabilities, upon answering a sensitive question the LLM will delete the original response in what Knostic calls a “take back” action. Knostic researchers observed the LLM having “second thoughts” as it provided a new answer after deleting the first. The “take back” behavior can be exploited to force the LLM to respond and divulge prohibited matters before it takes back the original text.
Notably, the Knostic Research team succeeded in using this method to extract the system prompt of Microsoft Copilot for Microsoft 365.
The findings from Knostic also highlighted the importance of designing and testing LLM applications and agentic systems with an approach that goes beyond evaluating just the model and prompts. Such an approach should be complex, consisting of multiple components such as guardrails, web interfaces and backend processes.
Knostic is a venture capital-backed startup that has raised one round of $3.3 million. Investors in the company include Pitango Venture Capital Ltd., DNX Ventures, Seedcamp Investments Ld. and Shield Capital Partners.
Support our open free content by sharing and engaging with our content and community.
Where Technology Leaders Connect, Share Intelligence & Create Opportunities
SiliconANGLE Media is a recognized leader in digital media innovation serving innovative audiences and brands, bringing together cutting-edge technology, influential content, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — such as those established in Silicon Valley and the New York Stock Exchange (NYSE) — SiliconANGLE Media operates at the intersection of media, technology, and AI. .
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a powerful ecosystem of industry-leading digital media brands, with a reach of 15+ million elite tech professionals. The company’s new, proprietary theCUBE AI Video cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.