Prompt Template Benchmark for Qwen3.8 Flash Next on SWE-bench Verified
An empirical benchmark evaluated three prompt templates (Stock, Fixed, and Sharp) across medium and extra-high reasoning effort levels for Qwen3.8 Flash Next on 100 SWE-bench Verified tasks. Results showed that while the Sharp template led in resolution rate (94%) at medium reasoning effort, the default Stock template achieved the highest resolution rate (99%) when scaled to extra-high effort. The comparison shows that specialized prompt templates like Sharp offer efficient speed and accuracy at lower reasoning budgets, but basic templates perform better when allowed higher token budgets. This provides practical guidelines for local LLM developers balancing execution latency against task accuracy. The test benchmarked RadixArk's Qwen3.8-Flash-Next-NVFP4 quantized model using mini-SWE-agent 2.4.6 on an RTX PRO 6000 WS GPU with a 262K context window. Increasing reasoning effort from medium to extra-high yielded no accuracy improvement for the Sharp template (94% to 94%) despite a 53.4% wall time increase, whereas Stock jumped from 91% to 99%.
## BACKGROUND
SWE-bench is an automated benchmark that evaluates LLMs on their ability to solve real-world GitHub software issues. mini-SWE-agent is a lightweight open-source framework designed to automate software engineering agent workflows efficiently. NVFP4 is NVIDIA's low-precision 4-bit floating-point quantization format optimized for high-performance model inference with minimal loss in dynamic range.