Anthropic and Academic Partners Introduce CryptanalysisBench to Evaluate LLMs

Anthropic, in collaboration with researchers from ETH Zurich, Tel Aviv University, and the University of Haifa, has introduced CryptanalysisBench. This new benchmark is designed to evaluate the cryptanalysis capabilities of large language models (LLMs). As LLMs become more advanced, understanding their ability to decipher cryptographic systems is critical for AI safety and cybersecurity. This benchmark helps researchers systematically measure whether AI models pose risks to modern encryption standards or can assist in security audits. CryptanalysisBench consists of 191 tasks spanning six cryptographic primitive families, including hash functions, block ciphers, AEADs, KEMs, PKEs, and digital signatures. These tasks are drawn from four NIST cryptography competitions alongside widely-studied academic and production ciphers.