OpenAI Triples GPT-5.6 Sol's ARC-AGI-3 Score by Adjusting API Settings
OpenAI announced that enabling two specific API settings tripled the performance of its GPT-5.6 Sol model on the ARC-AGI-3 benchmark. The adjustment resolved a memory constraint in the evaluation harness, allowing the model to retain reasoning and compact context over time. This highlights that AI benchmark scores do not measure models in isolation, but rather the entire system including API settings, harness design, and prompting. It shows how memory retention and context management are critical for LLMs to solve complex, interactive reasoning tasks. While GPT-5.6 Sol was capable of solving open mathematical problems, it initially struggled with the 2D puzzle games of ARC-AGI-3 because the evaluation harness prevented it from remembering what it had learned. Enabling settings that allow the model to retain reasoning and compact context resolved this bottleneck.
## BACKGROUND
The Abstraction and Reasoning Corpus for AGI (ARC-AGI) is a benchmark designed to measure fluid intelligence and novel abstract reasoning through visual grid puzzles. An evaluation harness is a testing framework used to run models against standardized datasets and prompts to measure their performance.