Benchmarking Local LLMs and Quantization Levels with SVG Pelican Generation

A community developer tested several large open-source LLMs quantized by Unsloth on a 128GB Apple Silicon system using Simon Willison's famous "pelican riding a bicycle" SVG prompt. The evaluation revealed that Qwen3.8 Flash-Next produced detailed vector outputs, while 2-bit quantizations (~12GB) proved sufficient for simple SVG tasks and 4-bit quantizations held up well for complex coding. This testing provides practical guidance for AI enthusiasts running local hardware on how low quantization can go before code output degrades. It demonstrates that users can achieve surprisingly strong visual code generation on consumer setups without needing high-precision, memory-intensive model weights. While Qwen3.8 Flash-Next and Qwen3.8 27B performed strongly, DeepSeek V4 Flash delivered unexpectedly underwhelming results. Cross-referencing with standard benchmarks like Terminal-Bench 2.1, GPQA Diamond, and IFBench confirmed that 4-bit quantized models retain high accuracy, whereas 1-bit variants collapse significantly.