Terence Tao Warns Against Delegating Critical Thinking to AI

Fields Medalist Terence Tao warned that humans risk losing comprehension and control over critical real-world systems by increasingly delegating problem-solving and thinking processes to AI. Speaking at the Science x AI Summit on October 16, he urged the scientific community to transition toward a 'Math 2.0' paradigm where AI enhances human understanding rather than bypassing it. This warning highlights critical safety and comprehension concerns as AI systems are increasingly deployed in high-stakes domains like electric grids, air traffic control, and drug discovery. It emphasizes that retaining human oversight, verification, and critical thinking is essential to prevent catastrophic failures in real-world applications. Tao noted that while pure mathematics risks are mitigated by formal verification tools like proof assistants, applying unverified AI solutions to complex real-world infrastructure is extremely hazardous. He stressed that AI models might exploit loopholes or boundary conditions to meet surface-level metrics while deviating entirely from human intentions.