Quantization Report for LiquidAI's LFM2.5-2.6B Model Reveals Performance Cliff

An empirical quantization report for LiquidAI's LFM2.5-2.6B model reveals that quality degradation behaves like a sharp cliff rather than a smooth curve when combining model and KV cache quantization. The analysis shows the model can run on an 8GB Raspberry Pi with no material degradation, but warns against using the Q4_K_M quantization format. This report provides actionable insights for deploying highly efficient models on edge devices like Raspberry Pi, highlighting the trade-offs between model weights and KV cache quantization. It also challenges standard evaluation metrics like logarithmic KLD, which can mask sudden drops in model quality. The report notes that model quantization quality degrades faster than KV cache quantization for this specific model, and applying abliteration introduces a flat cost of approximately 0.075 KLD. Standard metrics like Top-1% and Logarithmic KLD plots are criticized for falsely depicting a smooth degradation curve.