FreedomIntelligence Releases HuatuoGPT-3-27B with SFT-Free OnePO Training

FreedomIntelligence has released HuatuoGPT-3-27B, a medical large language model fine-tuned using a novel technique called One-stage Policy Optimization (OnePO). Alongside the model weights, the team open-sourced the complete training code, a medical reinforcement learning dataset, and an 8B rubric grader. OnePO challenges the conventional LLM training pipeline by completely bypassing domain-specific Supervised Fine-Tuning (SFT) in favor of direct reinforcement learning. This approach simplifies domain adaptation for complex fields like medicine while enabling models to learn effectively through dynamic teacher guidance. Built on top of a Qwen 27B base model, HuatuoGPT-3 uses transient outputs from a teacher model to bootstrap medical domain knowledge rapidly during early training. As the model improves, teacher guidance is automatically retired to transition into pure on-policy RL exploration, preventing gradient starvation.