432 bilingual prompts, four to twelve text regions each, scored by a vision-language model on fidelity, clarity, placement and scene. The results separate sharpness from correctness.
Benchmark
UltraText Bench
Whether an image generator can render dense visual text — long strings spread across several regions of one picture, placed where the prompt asks and legible at the end. It has 432 human-reviewed prompts across 24 real-world scene categories and three difficulty levels, split equally between English and Chinese. Each prompt specifies the exact strings for four to twelve separate text regions.
- Maintainer
- Deyuan Liu and colleagues
- Score
- composite
- Last checked
- October 8, 2026
Leaderboard
No verified results recorded yet.
Methodology
Generation is prompt-only: no reference image, no layout mask, no editing pass. Every prompt carries a structured reference describing each region's content, placement and visual attributes, and the Q-Judger vision-language model scores the produced image against that reference on four dimensions — text fidelity, text clarity, spatial quality and scene quality. The authors report 24 model configurations and ran a human evaluation of the automatic scores with ten participants.
Important limitations
The scorer is itself a model, so its judgements inherit the limits of a vision-language system reading small text. Only English and Chinese are covered, which leaves out the scripts where rendering is hardest. The benchmark is a preprint rather than a maintained public leaderboard, and the prompts are fixed, so scores will mean less once they are in training data.