Evaluating Small Active-Parameter LLMs on Tool Calling Instead of Parametric Knowledge

A developer shared a shift in evaluating small active-parameter models, arguing they should be judged on their ability to call tools and retrieve external data rather than their internal knowledge. The author notes that while these models often hallucinate obscure facts, they can excel at leveraging external APIs, codebases, and documentation when prompted correctly. This perspective shifts the focus of local LLM deployment from massive, resource-heavy models with deep parametric knowledge to smaller, efficient models optimized for tool-use and Retrieval-Augmented Generation (RAG). It highlights a practical path for deploying AI agents on consumer hardware by relying on external databases rather than static, hard-to-update model weights. A major limitation of this approach is that small models must still possess enough self-awareness to recognize when they do not know an answer, otherwise they will confidently hallucinate instead of triggering a tool call. The author suggests that models should be explicitly trained or fine-tuned to bail out to external tools when confidence is low, rather than relying solely on standard benchmarks like MMLU.