Data Is Better Together: A Look Back and Forward

For the past few months, we have been working on the Data Is Better Together initiative. With this collaboration between Hugging Face and Argilla and the support of the open-source ML community, our goal has been to empower the open-source community to create impactful datasets collectively.

Now, we have decided to move forward with the same goal. To provide an overview of our achievements and tasks where everyone can contribute, we organized it into two sections: community efforts and cookbook efforts.

Community efforts

Our first steps in this initiative focused on the prompt ranking project. Our goal was to create a dataset of 10K prompts, both synthetic and human-generated, ranked by quality. The community's response was immediate!

• In a few days, over 385 people joined.
• We released the DIBT/10k_prompts_ranked dataset intended for prompt ranking tasks or synthetic data generation.
• The dataset was used to build new models, such as SPIN.

Seeing the global support from the community, we recognized that English-centric data alone is insufficient, and there are not enough language-specific benchmarks for open LLMs. So, we created the Multilingual Prompt Evaluation Project (MPEP) with the aim of developing a leaderboard for multiple languages. For that, a subset of 500 high-quality prompts from DIBT/10k_prompts_ranked was selected to be translated into different languages.

• More than 18 language leaders created the spaces for the translations.
• Completed translations for Dutch, Russian or Spanish, with many more efforts working towards complete translations of the prompts.
• The creation of a community of dataset builders on Discord

Going forward, we’ll continue to support community efforts focused on building datasets through tools and documentation.

Cookbook efforts

As part of DIBT, we also created guides and tools that help the community build valuable datasets on their own.