Japanese Stable Diffusion

Stable Diffusion, developed by CompVis, Stability AI, and LAION, has generated a great deal of interest due to its ability to generate highly accurate images by simply entering text prompts. Stable Diffusion mainly uses the English subset LAION2B-en of the LAION-5B dataset for its training data and, as a result, requires English text prompts to be entered producing images that tend to be more oriented towards Western culture.

rinna Co., Ltd. has developed a Japanese-specific text-to-image model named "Japanese Stable Diffusion" by fine-tuning Stable Diffusion on Japanese-captioned images. Japanese Stable Diffusion accepts Japanese text prompts and generates images that reflect the culture of the Japanese-speaking world which may be difficult to express through translation.

In this blog, we will discuss the background of the development of Japanese Stable Diffusion and its learning methodology. Japanese Stable Diffusion is available on Hugging Face and GitHub. The code is based on 🧨 Diffusers.

Hugging Face model card: https://huggingface.co/rinna/japanese-stable-diffusion
Hugging Face Spaces: https://huggingface.co/spaces/rinna/japanese-stable-diffusion
GitHub: https://github.com/rinnakk/japanese-stable-diffusion

Stable Diffusion

Recently diffusion models have been reported to be very effective in artificial synthesis, even more so than GANs (Generative Adversarial Networks) for images. Hugging Face explains how diffusion models work in the following articles:

The Annotated Diffusion Model
Getting started with 🧨 Diffusers

Generally, a text-to-image model consists of a text encoder that interprets text and a generative model that generates an image from its output.

from Stable Diffusion with 🧨 Diffusers