Debating the Definition of Model Distillation and the Case for Open-Weight Models

A discussion on Reddit highlights the risks of relying on closed APIs amid Anthropic's legal challenges, urging a shift to self-hosted open-weight models. The author also argues that using API outputs for training data is mislabeled as "distillation," which technically requires access to a teacher model's logits. As enterprises face compliance, IP exposure, and data leakage risks with closed LLM vendors, migrating to self-hosted open-weight models becomes a viable alternative. However, this transition requires implementing strict runtime governance to prevent unauthorized code execution and data leaks. The author points out that true knowledge distillation relies on using full logits to transfer a teacher model's internal representations, which closed APIs like Claude do not provide. Therefore, using generated outputs to train new models is simply data generation, a practice even proprietary vendors likely use.