Simon Willison Explores LLM-Generated Web Audio Music with Scrimshaw Jukebox

Simon Willison demonstrated an experiment where Anthropic's Claude designed a custom text-based music notation format and built an interactive retro Web Audio synthesizer player called Scrimshaw Jukebox. The generated web artifact features six original adventure game tracks complete with editable text scores and interactive visual controls. This experiment highlights the expanding capabilities of Large Language Models to handle complex multi-modal creative tasks by bridging text representations, web application code, and browser-native audio synthesis. It demonstrates how LLMs can compose coherent multi-track music and construct functional web tools simultaneously from a single text prompt. The generated web synthesizer plays polyphonic tracks with up to 16 distinct voices—such as steel drum, marimba, flute, and synthesized percussion—using standard Web Audio APIs without relying on audio sample files. Users can edit the raw text score, mute individual instrument channels, adjust playback speed and looping, and watch a real-time piano-roll visualization.