DEVOURED
Qwen3.8 vs Qwen3.6 vs Gemma 4 on a 24GB GPU
Qwen3.8-27B outperforms Qwen3.6-27B and Gemma 4 31B in most coding and document tasks when run locally on 24GB of VRAM.
What: A side-by-side test on an RTX 4090 found that Qwen3.8-27B is the most reliable general-purpose model for coding and reasoning tasks, while Gemma 4 31B-it remains superior for structured tool-calling and vision tasks.
Why it matters: This highlights that model choice for local deployment is highly task-dependent, and improvements in model architecture often provide significant performance gains without requiring more hardware resources.
Takeaway: If you are building a local coding agent on an RTX 3090 or 4090, default to Qwen3.8-27B, but switch to Gemma 4 31B-it if your workflow relies heavily on strict, schema-bound tool usage.
Deep dive
- Coding reliability: Qwen3.8 passed 12/12 coding tasks compared to 8/12 for Qwen3.6 and 6/12 for Gemma 4.
- Memory headroom: At 64K context, Qwen models left ~4.2GB of VRAM free, while Gemma 4 required switching to Q8_0 KV-cache to avoid out-of-memory errors.
- Tool calling: Gemma 4 demonstrated perfect accuracy on single-call tool benchmarks, outperforming both Qwen variants.
- Speed: All three models performed similarly in terms of token throughput, with decode speeds around 45–49 tokens per second.
- Recommendation: Qwen3.8-27B is the recommended starting point for general agent work; Gemma 4 is a strong specialist alternative for vision and tool routing.
Decoder
- GGUF: A binary format used to store models for inference with CPU/GPU offloading, popularized by the llama.cpp project.
- Quantization: The process of reducing the precision of model weights (e.g., from 16-bit to 4-bit) to reduce memory usage at the cost of slight precision loss.
Original article
Full article content is not available for inline reading.
Explore all 43 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi