Model Cards

Introduction

Model cards are an important documentation framework for understanding, sharing, and improving machine learning models. When done well, a model card can serve as a boundary object, a single artefact that is accessible to people with different backgrounds and goals in understanding models - including developers, students, policymakers, ethicists, and those impacted by machine learning models.

Today, we launch a model card creation tool and a model card Guide Book, which details how to fill out model cards, user studies, and state of the art in ML documentation. This work, building from many other people and organizations, focuses on the inclusion of people with different backgrounds and roles. We hope it serves as a stepping stone in the path toward improved ML documentation.

In sum, today we announce the release of:

1. A Model Card Creator Tool, to ease card creation without needing to program, and to help teams share the work of different sections.

2. An updated model card template, released in the huggingface_hub library, drawing together model card work in academia and throughout the industry.

3. An Annotated Model Card Template, which details how to fill the card out.

4. A User Study on model card usage at Hugging Face.

5. A Landscape Analysis and Literature Review of the state of the art in model documentation.

Model Cards To-Date

Work to-date on documentation within ML has provided for different audiences. We bring many of these ideas together in the work we share today.

Our Work

Our work presents a view of where model cards stand right now and where they could go in the future. We conducted a broad analysis of the growing landscape of ML documentation tools and conducted user interviews within Hugging Face to supplement our understanding of the diverse opinions about model cards. We also created or updated dozens of model cards for ML models on the Hugging Face Hub, and informed by all of these experiences, we propose a new template for model cards.