Visualizing GPT-6 Astra's Capabilities via the Pelican SVG Benchmark
Simon Willison evaluated the newly released GPT-6 Astra model against GPT-5.6 variants across different reasoning effort levels using his signature 'pelican riding a bicycle' SVG generation prompt. The comparison revealed that Astra produces significantly higher quality vector graphics across all reasoning levels, with even its lowest setting outperforming GPT-5.6 Sol at maximum reasoning. The evaluation illustrates how newer model architectures can deliver superior spatial reasoning and code synthesis while consuming fewer tokens, keeping total task costs competitive despite higher per-token pricing. It provides practical insights into balancing model choice, reasoning effort parameters, and API expenses for visual code generation tasks. While Astra costs twice as much per token as GPT-5.6 Sol ($10/$50 per million tokens versus $5/$30), its reduced token output makes running tasks surprisingly cost-effective, such as generating a high-quality SVG at the low reasoning setting for 9.55 cents. However, Astra does not support disabling reasoning entirely (`reasoning=none`) and still struggles to correctly render anatomical details like pelican legs spanning both sides of the bicycle frame except at maximum effort.
## BACKGROUND
The 'Pelican Bicycle Benchmark' is an informal LLM benchmark created by Simon Willison in 2024 that tests a model's ability to generate raw Scalable Vector Graphics (SVG) code representing a physically challenging scene. Writing SVG code requires large language models to demonstrate complex spatial reasoning without real-time visual feedback. Additionally, the reasoning effort parameter controls how many internal thinking tokens a reasoning model generates before producing its final output.