PRX Part 3 — Training a Text-to-Image Model in 24h!

Introduction

Welcome back 👋

In the last two posts (Part 1 and Part 2), we explored a wide range of architectural and training tricks for diffusion models. We tried to evaluate each idea in isolation, measuring throughput, convergence speed, and final image quality, and tried to understand what actually moves the needle.

In this post, we want to answer a much more practical question:

What happens when we combine all the tricks that worked?

Instead of optimizing one dimension at a time, we’ll stack the most promising ingredients together and see how far we can push performance under a strict compute budget.

To make things concrete, we’re doing a 24-hour speedrun:

• 32 H200
• ~$1500 total compute budget (2$/hour/GPU)

This is very far from the early diffusion days, where training competitive models could cost millions of dollars. The goal here is to demonstrate how much the field has evolved and how far careful engineering can take you in just a single day of training.

This speedrun is not just a fun experiment. It will likely serve as the foundation for our large-scale training recipe going forward.

Alongside the results, we’re also open-sourcing our code (Github link), which contains:

• The training code used for this speedrun
• The experimental framework from the previous blog post

So you can reproduce, modify, and extend everything yourself.

The Training Recipe

Now let’s walk through what went into this 24h run.

X-prediction and Training in the Pixel Space

We use the x-prediction formulation from Back to Basics: Let Denoising Generative Models Denoise [Li and He, 2025]. As seen in Part 2, this enables training directly in pixel space and eliminates the need for a VAE altogether. We use a patch size of 32 and use a 256-dimensional bottleneck in the initial token projection layer. This design keeps the sequence length under control, making pixel-space training computationally manageable even at higher resolutions.

At 512px, the sequence length is:

(512/32)2=256 (512 / 32)^2 = 256