Hugging Face and JFrog partner to make AI Security more transparent

We are pleased to announce our partnership with JFrog, creators of the JFrog Software Supply Chain Platform, as part of our long-standing commitment to provide a safe and reliable platform for the ML community.

We have decided to add JFrog's scanner to our platform to continue improving security on the Hugging Face Hub. JFrog's scanner brings new functionality to scanning, aimed at reducing false positives on the Hub. Indeed, what we currently observe is that model weights can contain code that is executed upon deserialization and sometimes at inference time, depending on the format. This code is oftentimes a non harmful practicality for the developer. As our picklescan scanner only performs pattern matching on module names, we cannot always confirm that usage of a given function or module is malicious. JFrog goes a step deeper and will parse and analyze code it finds in models weights to check for potential malicious usage.

Interested in joining our security partnership / providing scanning information on the Hub? Please get in touch with us over at security@huggingface.co.

Model security refresher

To share models, we serialize weights, configs and other data structures we use to interact with the models, in order to facilitate storage and transport. Some serialization formats are vulnerable to nasty exploits, such as arbitrary code execution (looking at you pickle), making shared models that use those formats potentially dangerous.

As Hugging Face has become a popular platform for model sharing, we’d like to help protect the community from this, hence why we have developed tools like picklescan and why we are integrating JFrog in our scanner suite.

Pickle is not the only exploitable format out there, see for reference how one can exploit Keras Lambda layers to achieve arbitrary code execution. The good news is that JFrog catches both of these exploits and more in additional file formats – see their Model Threats page for up to date scanner information.