CodeGemma - an official Google release for code LLMs

CodeGemma is a family of open-access versions of Gemma specialized in code, and we’re excited to collaborate with Google on its release to make it as accessible as possible.🤗

CodeGemma comes in three flavors:

• A 2B base model specialized in infilling and open-ended generation.
• A 7B base model trained with both code infilling and natural language.
• A 7B instruct model a user can chat with about code.

We’ve collaborated with Google to ensure the best integration into the Hugging Face ecosystem. You can find the three open-access models ready to use on the Hub. Among the features and integrations being released, we have:

• Models on the Hub, with their model cards and licenses. There are versions for the transformers library, checkpoints for use with Google’s original codebases, and full-precision GGUF files that the community can quantize.
• Transformers integration
• Integration with Google Cloud
• Integration with Inference Endpoints
• Code benchmarks

Table of contents

• What is CodeGemma
  • Evaluation Results
  • Prompt format
• Using CodeGemma
  • Demo
  • Using Transformers
  • Integration with Google Cloud
  • Integration with Inference Endpoints
• Additional Resources

What is CodeGemma?

CodeGemma is a family of code-specialist LLM models by Google, based on the pre-trained 2B and 7B Gemma checkpoints. CodeGemma are further trained on an additional 500 billion tokens of primarily English language data, mathematics, and code to improve on logical and mathematical reasoning, and are suitable for code completion and generation.

This image is from the original report

Evaluation Results

Model	Pretraining size [tokens]	Python	JavaScript
10B+ models
Code Llama 13B	2,620B	50.6	40.92
Code Llama 13B	2,620B	42.89	40.66
7B models
CodeGemma 7B	500B	52.74	47.71
Code Llama 7B	2,620B	40.48	36.34
Code Llama 7B	2,620B	25.65	33.11

Here is a table from the original report with a breakdown per language.

Prompt format

Using CodeGemma

Demo

Using Transformers

pip install --upgrade transformers