Jeff-Qwen3.5-0.8B v1.2 Uses Modular LoRA Adapters for Fast LLM Decision Routing
Developer firelex released Jeff-Qwen3.5-0.8B v1.2 alongside 9 domain-specific LoRA adapters (~40 MB each) that serve as a fast "System 1" decision layer in front of larger LLMs like Qwen3.8-27B. By handling routine routing and classification tasks and passing only uncertain queries to the 27B model, the setup achieves 38× faster decision speeds and an 8.7-point boost in average accuracy while consuming under 2 GB of extra RAM. This modular approach demonstrates how combining small local models with lightweight LoRA adapters can dramatically accelerate agentic workflows while maintaining high accuracy. It highlights a pragmatic trend in LLM architecture toward smart model routing, enabling edge devices and local setups to offload heavy reasoning costs. Each adapter was trained by mixing in 10% of the base model's dataset to preserve general zero-shot classification capabilities. On five standard AI agent inbox tasks (guardrails, triage, intent, tool selection, and grounding), decision time dropped from 8.1 seconds to 0.25 seconds while accuracy improved from 87.7% to 95.7%.
## BACKGROUND
LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique that adds small, trainable layers to a frozen base model instead of retraining all parameters. LLM model routing is an architectural pattern where incoming queries are pre-evaluated by a fast classifier and sent to the most cost-effective or competent model for execution.