French artificial intelligence startup Mistral AI SAS today introduced a lightweight multimodal safety artificial intelligence open-weight model that can classify outputs for AI models that outperforms other large language models up to seven times its size, setting a new standard for moderation.

The new model, named Shieldstral, allows developers to write policies in natural language questions at runtime, and the model returns a safety score.

It requires no specialized retraining and stores positive capabilities for both text and images. It also provides a verdict in the form of a single token: a “yes” or a “no,” making the result completely unambiguous.

According to the company, the model has extremely strong text safety, matching or outperforming models that outweigh it by over seven times across diverse safety benchmarks, ranking an overall average safety score of 84.9%. It also achieves an average multimodal image safety benchmark of 83.8%, outperforming all evaluated baselines.

The model operates by having developers provide it with a single instruction, a high-level task carefully framing its purpose and evaluation context, e.g., “Evaluate the safety of advertisements, memes, and photos. Flag risky behavior, discrimination, privacy, and deception. Apply a strict standard.” Next, the user query, e.g., “Does this advertisement contain offensive or unsafe material?” Finally, the content, e.g., a picture of the advertisement or meme.

Mistral taught Shieldstral to discriminate, not merely memorize policies. This means that in its reasoning and classification performance, the model can apply two closely related but different policies and keep them separate in its “brain,” for example, “is this about malware,” or “is this about cybersecurity,” and acknowledge when one is violated and the other is not.

For malware content, Shieldstral can handle what’s called “contrastive pairs” like near-identical ransomware outputs: one version that walks a reader through writing and deploying their own malware, the other analyzes ransomware behavior so a defender can detect it. The model is trained to assign the first to a “malware instructions” policy (which is a violation) and the second to a “cybersecurity discussion” policy (which is okay), so at runtime it can apply a fine-grained distinction.

A user can ask: “Does this comply with the no malware instructions policy?”

A single natural language prompt covers text, images, and text-plus-image content across prompts, responses, and prompt-response pairs. Policies can be free-form queries and re-targeted at inference time, allowing users to easily customize and determine whether content outputs are safe, and to quickly determine whether inputs or outputs are safe.

This means that it can be used to quickly generate “yes” or “no” responses for customer service text safety, AI assistant refusal detection for dangerous requests, policy violations and image generation security.

At only 3 billion parameters, the model is also lightweight enough to run on a single 16-gigabyte graphics processing unit. This also means that it can run alongside a much larger model extremely quickly and efficiently to guardrail inputs or outputs with little extra delay, even in edge environments, where memory and compute bandwidth are scarce.

The company said Shieldstral will become a stepping stone toward moderation that can adapt to context instead of enforcing rigid taxonomic guesses onto conversations. This will allow for more natural multilingual coverage and longer-document coverage in the future, alongside broader multimodal safety.

Image: SiliconANGLE ChatGPT / Unsplash