“Llama 3.2 in Keras”

This is going to be the shortest blog post ever.

Question: Llama 3.2 landed two weeks ago on Hugging Face / Transformers. When will it be available in Keras?

Answer: It has been working from day 1 😀. There is nothing to wait for.

Yes, Keras Llama3 can be loaded from any standard (i.e. safetensors) Hugging Face checkpoint, including the 3.2 checkpoints. If a conversion is required, it happens on the fly. Try this:

!pip install keras_hub

from keras_hub import models.Llama3CausalLM
model = Llama3CausalLM.from_preset("hf://meta-llama/Llama-3.2-1B-Instruct", dtype="bfloat16")
model.generate("Hi there!")

Here is a Colab to try this out. Enjoy! 🤗

OK, OK, I'm being told that if I want to publish a blog post, I have to fill the space. Here are a couple of additional things to know about Keras.

Keras is multi-backend

Keras is the time-tested modeling library for JAX, PyTorch and TensorFlow. You might have noticed this line in the demo Colab:

import os
os.environ["KERAS_BACKEND"] = "jax" # or "torch", or "tensorflow"

It has to appear before import keras and controls if the model is running on JAX, PyTorch or TensorFlow. Very handy to try your favorite models on JAX with XLA compilation 🚀.

What is keras-hub?

Keras is a modeling library and keras-hub is its collection of pre-trained models. It was previously called KerasNLP and KerasCV. The rename is in progress. It has all the popular pre-trained models (Llama3, Gemma, StableDiffusion, Segment Anything, ...) with their canonical implementation in Keras.

LLMs in Keras come "batteries included"

I mean, "tokenizer included". model.generate() just works on strings:

model.generate("Hi there!")
> "Hi there! I'm looking for information on how to ...

Same thing for training. You can train on a set of strings directly:

model.fit(strings) # list or dataset of input strings

Chatting with an LLM

The conversation, once formatted in this way, can be fed directly to model.generate().