Democratizing AI Safety with RiskRubric.ai

Building trust in the open model ecosystem through standardized risk assessment

More than 500,000 models can be found on the Hugging Face hub, but it’s not always clear to users how to choose the best model for them, notably on the security aspects. Developers might find a model that perfectly fits their use case, but have no systematic way to evaluate its security posture, privacy implications, or potential failure modes.

As models become more powerful and adoption accelerates, we need equally rapid progress in AI safety and security reporting. We're therefore excited to announce RiskRubric.ai, a novel initiative led by Cloud Security Alliance and Noma Security, with contributions by Haize Labs and Harmonic Security, for standardized and transparent risk assessment in the AI model ecosystem.

Risk Rubric, a new Standardized Assessment of Risk for models

RiskRubric.ai provides consistent, comparable risk scores across the entire model landscape, by evaluating models across six pillars: transparency, reliability, security, privacy, safety, and reputation.

The platform's approach aligns perfectly with open-source values: rigorous, transparent, and reproducible. Using Noma Security capabilities to automate the effort, each model undergoes:

• 1,000+ reliability tests checking consistency and edge case handling
• 200+ adversarial security probes for jailbreaks and prompt injections
• Automated code scanning of model components
• Comprehensive documentation review of training data and methods
• Privacy assessment including data retention and leakage testing
• Safety evaluation through structured harmful content tests

These assessments produce 0-100 scores for each risk pillar, rolling up to clear A-F letter grades. Each evaluation also includes specific vulnerabilities found, recommended mitigations, and suggestions for improvements.

What we found (as of September 2025)

Let’s look at general trends:

Safety risk is the “swing factor” – but it tracks closely with security posture