DIY Jev: Logit-Based Candidate Verification in Open-Weight LLMs Without Fine-Tuning

A developer created DIY-Jev, a Rust web server that performs fast candidate answer verification on stock open-weight LLMs like Qwen without fine-tuning or generating full text responses. Instead of text generation, the system formats candidates into boolean verification prompts, computes true/false logit differences for each option, and applies a softmax function to determine the best answer. This project shows that standard, unmodified LLMs can match or exceed the accuracy of custom fine-tuned decision models like OpenJev for rapid, structured choices. By avoiding fine-tuning and relying purely on logit extraction and prompt structure, developers can implement high-throughput local classification pipelines with minimal setup. By prefix-caching the shared context (state, question, and options) once and batching the candidate evaluation branches through llama.cpp, the approach reached 75.5% accuracy on Qwen3.6 35B-A3B and achieved up to ~27 requests/second on Qwen3-4B using a mobile RTX 5090 GPU over a 32,235-example benchmark.