Sunday, October 18, 2026
Herald of AI

AIGenerated AI news, written by AI and explained simply. Models, technology, culture and the people behind it.

New models

Compressed 27B Model Beats Full Version at Tool Calling in Under 8 GB

A heavily compressed version of Qwen3.8-27B runs on everyday hardware and outperforms the original on agent skills.

AIGenerated · Written by AI from the sources listed below

A tiny but punchy new open-weight model is turning heads in the local AI space. Underdog, the on-device assistant project from Conway Research, has released Saluki 27B under the Apache 2.0 license 1. The model takes Qwen3.8-27B — a dense 27-billion-parameter language model — and squashes it down to just 7.89 GB, compared to the original's 54 GB . That's roughly a seven-fold size cut.

What makes this compression special isn't just the small file size. Underdog tuned the quantization specifically to protect tool calling, the ability that turns a plain chat model into an AI agent that can use external tools 3. For developers who want to run a capable agent locally without a data-center GPU, that's a big deal 4.

How It Works

The model stacks three layers of work. First comes the base: Qwen3.8-27B from the Qwen3 team, a 64-layer model that mixes two types of attention and supports 262,144 tokens natively 3. Second is ISTA-DASLab's GSQ-RCO quantization, which learns accurate low-bit values for each part of the model and assigns the best compression type to every layer under a strict size budget . ISTA's own smallest version of this model came in at 8.4 GB 2.

Underdog then added its own tuning pass on top, shrinking the file to 7.89 GB and targeting tool-calling performance specifically. The resulting file is named IQ2-mix and carries an imatrix tag 3. The company has not published the full recipe for this final step 1.

Benchmarks: Tool Calling Shines, Math Takes a Hit

Underdog tested Saluki 27B against the full-size model using the same benchmark harness 2. On Underdog Bench — 120 tasks drawn from the Berkeley Function Calling Leaderboard v4 — Saluki scored 88, edging out the full model's 84 3. The gap was even wider on parallel tool calls: Saluki scored 42 versus the full model's 35, a 120 percent retention rate .

On the SWE-bench Verified coding test, the full model pulled ahead, fixing 33 of 50 issues compared to Saluki's 30 3. And the trade-offs show up clearly in math. On AIME 2025, Saluki scored 79.2 versus the full model's 96.7, a drop of about 17 points 2. Across nine benchmarks, the compressed model retained about 96 percent of the original's performance on average 1.

Running It Locally

One of Saluki 27B's biggest advantages is compatibility. It runs in stock llama.cpp, the widely used open-source runtime, with full GPU offload supported 1. That means anyone who can already run local models can use it — no custom forks or special builds needed 4. An optional vision add-on is available in two sizes: 629 MB or 928 MB, depending on whether you want to add image understanding 2.

The model is available on Hugging Face and on Underdog's launch page, released under Apache 2.0 so anyone can use, modify, and distribute it freely 1. A post from the Underdog AI X account highlighted the 7x size reduction and the tool-calling win, calling it the best model under 8 GB that runs in the Underdog assistant 5.

For developers building local AI agents who can't afford a $54 GB model download, Saluki 27B offers a compelling trade: a small footprint with stronger-than-expected agent skills and a direct path to running on consumer hardware.

Key facts

  • 7.89 GB file size, down from 54 GB for the full model
  • Beats the original at tool calling: 88 vs 84 on Underdog Bench
  • Parallel tool calls score 42, up from 35 in the full model
  • Runs on stock llama.cpp with no custom fork required
  • Released under Apache 2.0 by Underdog / Conway Research

Seen on social

Today we're releasing Underdog Saluki 27B Based on Qwen 3.8 27B, Saluki is almost 7x smaller than its full-precision counterpart while it keeps 96% of its benchmark performance and beats Qwen at tool calling It's the best model <8GB that runs in Underdog Saluki is available… pic.twitter.com/NNmNpjNW3r

— Underdog AI on X, see the post

Sources

  1. Meet the Underdog Saluki 27B: A 2-bit Qwen3.8-27B That Beats the Original at Tool Calling — marktechpost.com
  2. Meet the Underdog Saluki 27B: A 2-bit Qwen3.8-27B That Beats the ... — syncai.news
  3. Meet the Underdog Saluki 27B: A 2-bit Qwen3.8-27B That Beats the ... — oo.news
  4. Run Underdog Saluki 27B Locally with Llama.cpp: Full Setup Guide — mindstudio.ai
  5. Saluki 27B: 2-Bit Qwen Model Beats Original at Tool Calling — aidailypost.com

Related stories