# stabilityai/stable-diffusion-3.5-medium — vLLM serving recipe

> Stability AI's Stable Diffusion 3.5 text-to-image family (medium 2.5B, large 8.1B, large-turbo) via vLLM-Omni with Cache-DiT acceleration

Configuration: hardware `h200` (8× H200) · variant `default` (BF16) · task `t2i`

- Interactive page: https://vllm-recipes-nszo2aalz-inferact-inc.vercel.app/stabilityai/stable-diffusion-3.5-medium
- This configuration as JSON: https://vllm-recipes-nszo2aalz-inferact-inc.vercel.app/stabilityai/stable-diffusion-3.5-medium.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`, `large`, `large_turbo`
- `task`: `t2i`

## Install

Requires vLLM >= 0.12.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Extra dependencies:

- Pin vllm==0.33.0 for Stable Diffusion 3.5

```bash
uv pip install --torch-backend=cu130 vllm==0.33.0
```

- vllm-omni provides the image generation backend

```bash
uv pip install git+https://github.com/vllm-project/vllm-omni.git
```

## Serve

vLLM-Omni recipe — the generation endpoints and request examples are in the guide below.

```bash
vllm serve stabilityai/stable-diffusion-3.5-medium \
  --omni
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

Stable Diffusion 3.5 text-to-image generation models, served via **vLLM-Omni** with
optional **Cache-DiT** acceleration.

Supported variants:
- `stabilityai/stable-diffusion-3.5-medium` — 2.5B params
- `stabilityai/stable-diffusion-3.5-large` — 8.1B params
- `stabilityai/stable-diffusion-3.5-large-turbo` — 8.1B params (timestep-distilled for few-step inference)

### Prerequisites

- vLLM-Omni on top of vLLM 0.33.0
- `diffusers` library

### Installation

```bash
uv venv
source .venv/bin/activate
uv pip install --torch-backend=cu130 vllm==0.33.0
uv pip install git+https://github.com/vllm-project/vllm-omni.git
```

#### AMD ROCm (MI300X, MI325X, MI355X)

> The Dockerfile builds the frozen vLLM 0.33.0 source over its published
> bootstrap image. ROCm build and native model validation remain pending;
> this is not a claim that a final ROCm wheel is available.

```bash
uv venv --python 3.12
source .venv/bin/activate
git clone https://github.com/vllm-project/vllm-omni.git && cd vllm-omni && docker build -f docker/Dockerfile.rocm -t vllm-omni-rocm:v033-source .
```

### Python Usage

```python
from vllm_omni.entrypoints.omni import Omni

omni = Omni(model="stabilityai/stable-diffusion-3.5-medium")

images = omni.generate(
    prompt="a cat wearing sunglasses, cyberpunk style",
    negative_prompt="blurry, low quality",
    height=1024, width=1024,
    num_inference_steps=28,
    guidance_scale=7.5,
    num_outputs_per_prompt=2,
)
```

### CLI Usage

```bash
python examples/offline_inference/text_to_image/text_to_image.py \
  --model stabilityai/stable-diffusion-3.5-medium \
  --prompt "a cat wearing sunglasses, cyberpunk style" \
  --negative-prompt "blurry, low quality" \
  --height 1024 --width 1024 \
  --num-inference-steps 28 \
  --guidance-scale 7.5
```

### Cache-DiT Acceleration

Enable caching for significant speed-ups:

```python
omni = Omni(
    model="stabilityai/stable-diffusion-3.5-medium",
    cache_backend="cache_dit",
    cache_config={
        "Fn_compute_blocks": 8,
        "Bn_compute_blocks": 0,
        "max_warmup_steps": 4,
        "residual_diff_threshold": 0.12,
    },
)
```

### Key Parameters

| Parameter | Default | Description |
|-----------|---------|-------------|
| `height` | 1024 | Image height (multiples of 16) |
| `width` | 1024 | Image width (multiples of 16) |
| `num_inference_steps` | 28 | Denoising steps |
| `guidance_scale` | 1.0 | Classifier-free guidance scale |

### References

- [SD3.5 Medium](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium)
- [SD3.5 Large](https://huggingface.co/stabilityai/stable-diffusion-3.5-large)
- [SD3.5 Large Turbo](https://huggingface.co/stabilityai/stable-diffusion-3.5-large-turbo)
- [Cache-DiT Acceleration](https://github.com/vipshop/cache-dit)