Debunking Misconceptions About Model Distillation in Open-Source LLMs

A community discussion highlights that training open-source models on synthetic text completions from public APIs is technically distinct from true knowledge distillation, which requires direct access to the teacher model's probability logits. Clarifying this distinction helps the AI community accurately evaluate the capabilities of open-source models, challenging the assumption that their performance is merely copied from proprietary systems. True distillation transfers knowledge using logit distributions, whereas public APIs only provide text completions that have already been filtered by safety guardrails and refusals.