SyncAI.news, a Varaisys broadcasting
Featuring Every Eval Ever Results on Hugging Face Model Pages
HF

Hugging Face Blog

· 1 min read

AI LabsHugging Face Blog

Featuring Every Eval Ever Results on Hugging Face Model Pages

Every Eval Ever (EEE) and Hugging Face Community Evals are now intercompatible. We enable cross-posting and interpreting evaluation results, while linking to open models, leaderboards, and a unified standardized metadata store.

EEE launched in February 2026 as a project of the EvalEval Coalition, the first cross-institutional effort to improve how AI evaluation results get reported by both first and third party evaluators. Hugging Face launched Community Evals in February 2026 to decentralize how benchmark scores get reported on the Hub. Combined, they patch gaps in how users, researchers, and policymakers trust, understand, and choose evaluations and models.

Evaluation results are how we measure model capabilities, compare models against each other, and reason about safety and governance, and yet they are scattered and hard to compare. They live in papers, leaderboards, blog posts, and harness logs, among others, each in its own format. The same model on the same benchmark often returns different scores depending on who ran it and how; LLaMA 65B, for one, has been reported at both 63.7 and 48.8 on MMLU. These gaps can arise from evaluation settings that we found are commonly unreported.

EEE is our fix for the reporting side. It's one JSON schema for an evaluation result that records:

  • who ran it
  • which model
  • how it was accessed
  • generation settings
  • what the metric actually means
  • [recommended] companion JSONL file for per-sample outputs.

Learn more about the schema and how to contribute here.

Now, it comes with better integration and attribution. Contributors can now send EEE results to Hugging Face Community Evals. We built a converter that takes your EEE records and writes the small YAML files Hugging Face expects, so you don't have to keep the same result in two formats by hand.

How Hugging Face Community Evals works together with EvalEval

Hugging Face Community Evals has two sides.

Here is what one of these leaderboards looks like:

Original source

This story was published by Hugging Face Blog. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on huggingface.co

Similar News