Arena Analyzes 3M+ Votes on Image Generation Evaluation Categories

Arena analyzed over 3 million user votes to evaluate the independence of its seven image generation categories. The analysis revealed significant overlap between some categories, such as Photorealistic and Portraits at 64%, but almost no overlap (under 1%) between Photorealistic and Art. This empirical data helps benchmark designers understand how users evaluate AI-generated images and highlights the need to refine evaluation categories. It shows that while some tasks require distinct benchmarks, others could potentially be consolidated due to high overlap. The study also found a 36% overlap between the Commercial Design and Text Rendering categories. These findings suggest that user preferences and model performance in image generation are highly task-dependent rather than uniform across all styles.