Welcome GPT OSS, the new open-source model family from OpenAI!

GPT OSS is a hugely anticipated open-weights release by OpenAI, designed for powerful reasoning, agentic tasks, and versatile developer use cases. It comprises two models: a big one with 117B parameters (gpt-oss-120b), and a smaller one with 21B parameters (gpt-oss-20b). Both are mixture-of-experts (MoEs) and use a 4-bit quantization scheme (MXFP4), enabling fast inference (thanks to fewer active parameters, see details below) while keeping resource usage low. The large model fits on a single H100 GPU, while the small one runs within 16GB of memory and is perfect for consumer hardware and on-device applications.

To make it even better and more impactful for the community, the models are licensed under the Apache 2.0 license, along with a minimal usage policy:

We aim for our tools to be used safely, responsibly, and democratically, while maximizing your control over how you use them. By using gpt-oss, you agree to comply with all applicable law.

According to OpenAI, this release is a meaningful step in their commitment to the open-source ecosystem, in line with their stated mission to make the benefits of AI broadly accessible. Many use cases rely on private and/or local deployments, and we at Hugging Face are super excited to welcome OpenAI to the community. We believe these will be long-lived, inspiring and impactful models.

Contents

• Introduction
• Overview
• API access through Inference Providers
• Local Inference
  • Using transformers
    • Flash Attention 3
    • Other optimizations
    • AMD ROCm support
    • Summary of Optimizations
  • llama.cpp
  • vLLM
  • transformers serve
• Fine Tuning
• Deploy on Hugging Face Partners
  • Azure
  • Dell
• Evaluating the Model
• Chats and Chat Templates
  • System and Developer Messages
  • Tool use with transformers

Overview of Capabilities and Architecture

Architecture

API access through Inference Providers

Local Inference

Using Transformers

pip install --upgrade transformers kernels accelerate "triton>=3.4"