@ClementDelangue: Number 5 trending on the hub! https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted…

X AI KOLs Following Models

Summary

A LoRA adapter for Qwen3.8-27B, trained with reinforcement learning to power the pi coding agent, tops the Hugging Face hub trends — matching the base model's xhigh thinking accuracy on SWE-bench Pro and Terminal-Bench 2.1 with about 4.5× fewer tokens.

Number 5 trending on the hub! https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted…
Original Article
View Cached Full Text

Cached at: 11/19/26, 05:33 AM

Number 5 trending on the hub! https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted…


harness-hub/Qwen3.8-27B-pi-adapted · Hugging Face

Source: https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted

A LoRA adapter forQwen/Qwen3.8-27B, trained with reinforcement learning to work as the model behind thepicoding agent on software-engineering and terminal tasks.

**Maximum-effort thinking accuracy, at a quarter of the tokens.**With thinking off, this adapter matches the base model thinking at its highest reasoning effort (xhigh): 64.7% vs 64.7% on SWE-bench Pro, 56.6% vs 54.2% on Terminal-Bench 2.1, while generating about 4.5× fewer tokens per task. Keep thinking off: turning it on for the adapter adds nothing. SeeThinking vs no thinking.

  • **SWE-bench Pro (hard 51):**39.2% → 64.7% (+25.5 points; 15 tasks newly solved, 2 lost).
  • **Terminal-Bench 2.1:**39.5% → 55.8% (+16.3 points; 16 newly solved, 2 lost).
  • **Thinking-level accuracy, without thinking:**matches the base model at its highest reasoning effort, with about 4.5× fewer generated tokens.
  • **Less work per task:**under half the turns of the base model on the tasks both solve.

All results are under the pi harness: one attempt per task, a 100-step budget, thinking off unless stated, text only (the vision tower is not served), sampling at temperature 0.8 with top-p 1.0 (the training sampling). Base model and adapter always run under identical settings. These numbers are not comparable with the ones on theQwen3.8-27B card, which uses other harnesses (Terminus, Claude Code), thinking at maximum effort, larger budgets and the full SWE-bench Pro.

Full recipe will be shared inthe release blogpostfor anyone to build their own harness adapter.

Pick your setup

ResultstransformersvLLMllama.cppLlama.appGGUFMergeTraining

A real SWE-bench Pro task under pi: the base model searches and re-runs snippets until it hits the 100-step limit; this adapter finds the function, edits it, runs the tests and fixes the last case, in 8 tool calls.

Still version, and the task

Task:instance\_ansible\_\_ansible\-d62496fe416623e88b90139dc7917080cb04ce70from SWE-bench Pro (hard 51), fixhuman\_to\_bytes. Both runs are from the evaluation above: the base model was graded 0, this adapter 1. Each line is one tool call as the trajectory recorded it, cut to fit.

https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted#resultsResults

BenchmarkQwen3.8-27BQwen3.8-27B,xhighthinkingQwen3.8-27B-opencode-adaptedQwen3.8-27B-opencode-adapted,xhighthinkingSWE-bench Pro (hard 51)39.2%64.7%**64.7%**66.7%Terminal-Bench 2.141.0%54.2%**56.6%**56.6% On Terminal-Bench 2.1 the four runs are compared on the 83 tasks all of them graded, so the base model and adapter scores differ slightly from the paired results below, which use the 86 tasks those two graded. “Thinking” is the chat template’s thinking mode at its default reasoning effort (xhigh).

Full paired results(intervals and sign tests against the base model)One attempt per task, thinking off, paired against the base model under pi on the same tasks. The interval is 95%; the p-value is an exact sign test on the tasks that changed outcome.

BenchmarkTasksQwen3.8-27BQwen3.8-27B-pi-adapted (step 160)Difference (points)Newly solved / lostSWE-bench Pro (hard 51)5139.2%64.7%+25.5 ± 14.215 / 2 (p = 0.0023)Terminal-Bench 2.18639.5%55.8%+16.3 ± 9.116 / 2 (p = 0.0013)Terminal-Bench 2.1, the 68 tasks not used to pick the checkpoint6839.7%55.9%+16.2 ± 10.613 / 2 (p = 0.0074)

  • **SWE-bench Pro (hard 51):**the 51 hardest tasks ofSWE-bench Pro, 2 h and 100 agent steps per task.
  • **Terminal-Bench 2.1:**all 89 tasks ofTerminal-Bench 2.1, 1 h and 100 agent steps per task.

Step 160 was picked by looking at the checkpoints’ scores on SWE-bench Pro and on an 18-task subset of Terminal-Bench 2.1, tracked during training.

Work per solved task(turns, visible content, reasoning and generated tokens)Medians per task, this adapter / the base model, over the tasks both solved: same tasks, same outcome, so the numbers measure only how much work each needed. Generated tokens are everything the model writes (visible content, reasoning and tool calls); content is the visible text between tool calls.

TasksTurnsContent tokensReasoning tokensGenerated tokensvs the base modelSWE-bench Pro1823 / 593.5k / 0.6k0 / 06.9k / 9.6kTerminal-Bench 2.1328 / 18n/a0 / 01.5k / 2.9kvs the base model with thinkingSWE-bench Pro2322 / 503.5k / 1.0k0 / 26.9k5.1k / 34.7kTerminal-Bench 2.1358 / 180.9k / 0.8k0 / 9.9k1.4k / 11.6k

  • Against the base model, the adapter writes short reasoning between tool calls (about 160 tokens per turn against about 10) and needs less than half the turns.
  • Against the base model with thinking, nearly all the difference is reasoning: the adapter reaches the same answers without it.
  • Content for the base model on Terminal-Bench 2.1 is not available: its run’s traces were not kept.

Thinking vs no thinkingSWE-bench Pro (hard 51)Terminal-Bench 2.1This adapter vs the base model with thinking (tasks only one solves)10 / 1012 / 10Generated tokens per task (median), base model with thinking / this adapter34.4k / 7.9k16.5k / 3.7kModel calls per task (median), base model with thinking / this adapter48 / 2119 / 9This adapter with thinking vs without (tasks only one solves)10 / 99 / 9 Thinking mode on its own brings the base model to this adapter’s accuracy; the adapter gets there without it, at a fraction of the tokens and model calls. Thinking runs use the template’s default reasoning effort (xhigh) and up to 32k output tokens per model call (16k on SWE-bench Pro, pi’s default at the time); fewer than 0.5% of turns reached that limit, as often for the adapter as for the base model.

Evaluation checks- **Correction.**An earlier version of this card reported 29.4% → 62.7% on SWE-bench Pro. Nine of the base model’s rollouts there had failed because the public tunnel between the sandboxes and the model server went down mid-run. We now check every rollout for this and reran all affected ones, on both sides; the numbers above are the corrected ones.

  • **Test files.**Agents, the base model and this adapter alike, sometimes write or edit test files while working, mostly their own scratch scripts to check a change. This cannot affect a grade: SWE-bench Pro’s grader resets all test files and applies the official tests before running them, and Terminal-Bench’s tests are copied in only at grading.
  • **Hidden reasoning.**The adapter’s visible reasoning is plain text:<think\>tags appear in its output in fewer than 1 turn in 1,000, and with thinking off the template never opens a thinking block.

https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted#use-with-piUse with pi

Serve the adapter with vLLM and point pi at it### https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted#1-serve-the-adapter-with-vllm1. Serve the adapter with vLLM

vllm serve Qwen/Qwen3.8-27B \
    --enable-lora --max-lora-rank 32 \
    --lora-modules qwen3.8-pi=harness-hub/Qwen3.8-27B-pi-adapted \
    --language-model-only --enable-prefix-caching --mamba-cache-mode align \
    --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3 \
    --default-chat-template-kwargs '{"enable_thinking": false}' \
    --max-model-len 131072

This is the configuration of the evaluations. Keep thinking off, as in training: with it on, the adapter is no more accurate and generates about 4× more tokens. Prefix caching on this hybrid model needs\-\-mamba\-cache\-mode align; without it, every agent turn re-reads its whole history.

https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted#2-point-pi-at-it2. Point pi at it

Install pi (Node.js 22.19 or newer):

npm install -g --ignore-scripts @earendil-works/pi-coding-agent

Add the server to~/\.pi/agent/models\.json:

{
  "providers": {
    "local": {
      "baseUrl": "http://localhost:8000/v1",
      "api": "openai-completions",
      "apiKey": "dummy",
      "models": [{ "id": "qwen3.8-pi" }]
    }
  }
}

"api": "openai\-completions"matters: without it pi uses the Responses API, which this adapter was not trained or evaluated with.

pi --provider local --model qwen3.8-pi --print "Find out why the tests in tests/test_parser.py fail and fix the bug."

https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted#use-with-pi-transformers-serveUse with pi (transformers serve)

Serve with transformers and point pi at ittransformers serveis a lightweight OpenAI-compatible server, for local use and experimentation (vLLM is faster under load). It handles Qwen’s tool calls out of the box. Serve the merged checkpoint fromMerge the adapter into the base model:

pip install "transformers[serving]"
transformers serve ./Qwen3.8-27B-pi-merged --dtype bfloat16 --continuous-batching --reasoning off

\-\-reasoning offkeeps thinking off, as in training. The server listens onhttp://localhost:8000, so the pi configuration fromthe vLLM sectionworks as is, with the modelidset to the name the server serves, here\./Qwen3\.8\-27B\-pi\-merged:

pi --provider local --model ./Qwen3.8-27B-pi-merged --print "Find out why the tests in tests/test_parser.py fail and fix the bug."

https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted#use-with-pi-llamacppUse with pi (llama.cpp)

Serve the GGUF adapter with llama.cpp and point pi at itThe GGUF version of Qwen3.8-27B-pi-adapted is inharness-hub/Qwen3.8-27B-pi-adapted-GGUF. It runs on top of a GGUF of the base model:

hf download ggml-org/Qwen3.8-27B-GGUF Qwen3.8-27B-Q8_0.gguf --local-dir .
hf download harness-hub/Qwen3.8-27B-pi-adapted-GGUF Qwen3.8-27B-pi-lora-F16.gguf --local-dir .

llama serve -m Qwen3.8-27B-Q8_0.gguf --lora Qwen3.8-27B-pi-lora-F16.gguf \
    --chat-template-kwargs '{"enable_thinking": false}' -c 65536 --port 9931 --alias qwen3.8-pi

Then point pi at it with thepi-llamaextension:

pi install git:github.com/huggingface/pi-llama
pi --model llama-cpp/qwen3.8-pi

The adapter was trained on the BF16 weights: the closer the base GGUF is to them, the closer you get to the trained behavior.Q8\_0is a good default; lower-bit quants drift further. The GGUF version has not been re-evaluated.

https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted#run-it-with-llamaapp-macosRun it with Llama.app (macOS)

Load the GGUF adapter on top of a Qwen3.8-27B GGUF1. InstallLlama.appand launch it once. On first launch it creates~/\.config/llama/models\.user\.ini. 2. Download the adapter and register the model: LORA=$(hf download harness-hub/Qwen3.8-27B-pi-adapted-GGUF Qwen3.8-27B-pi-lora-F16.gguf) cat >> ~/.config/llama/models.user.ini <<EOF [harness-hub/Qwen3.8-27B-pi] hf-repo = unsloth/Qwen3.8-27B-GGUF:Q4_K_XL ; This is just an example but you can also use any GGUF that is derived from Qwen3.8-27B lora = $LORA temp = 0.7 ctx-size = 32768 EOF hf\-reposets the base model: the app downloads it on first use, about 17 GB. To use another quant of the same base, change the part after the colon, for example:Q8\_0. 3. Restart Llama.app and chooseharness-hub/Qwen3.8-27B-pifrom the model list.

You can also use it through the app’s OpenAI-compatible server. Llama.app shows the port; the example below uses 9931:

curl localhost:9931/v1/chat/completions -d '{"model":"harness-hub/Qwen3.8-27B-pi","messages":[{"role":"user","content":"hi"}]}'

The adapter must be loaded on top of Qwen3.8-27B. It does not work with other base models.

https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted#merge-the-adapter-into-the-base-modelMerge the adapter into the base model

Get a standalone model, with no LoRA at serving timeThe adapter can be merged into the base weights with PEFT. The result is a regular Qwen3.8-27B checkpoint (about 54 GB in BF16) that any engine serves without LoRA support. Merge on CPU if the GPU is needed for serving afterwards:

import torch
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoTokenizer

base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.8-27B", dtype=torch.bfloat16, device_map="cpu")
model = PeftModel.from_pretrained(base, "harness-hub/Qwen3.8-27B-pi-adapted").merge_and_unload()
model.save_pretrained("Qwen3.8-27B-pi-merged")
AutoTokenizer.from_pretrained("harness-hub/Qwen3.8-27B-pi-adapted").save_pretrained("Qwen3.8-27B-pi-merged")

Then serve it like the base model, without the LoRA flags:

vllm serve Qwen3.8-27B-pi-merged --served-model-name qwen3.8-pi \
    --language-model-only --enable-prefix-caching --mamba-cache-mode align \
    --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3 \
    --default-chat-template-kwargs '{"enable_thinking": false}' --max-model-len 131072

The merged checkpoint can also be converted to GGUF and served with llama.cpp the same way.

https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted#trainingTraining

Method, data, hyperparameters and hardwareMethodAsyncGRPO(TRL, experimental), LoRAEnvironmentHarbortasks served byOpenEnv, rollouts in HF sandboxes, tool calls screened bysafeagentsHarnesspiTasks750scale800tasks interleaved with 300TermGradetasks; 1,919 task groups trained up to this checkpointRewardthe task verifier’scorrectness(falls back toreward), plus a tool-efficiency bonus of weight 0.1Steps160 (this checkpoint); the run was stopped at step 191 of 1000 plannedLearning rate2e-5LoRAr = 32, alpha = 64, on q, k, v, o, gate, up and down projectionsRollouts8 per task, temperature 0.8, thinking off, up to 50 agent steps and 1,800 s per rolloutBatch16 micro-batches of up to 65,536 tokens per step, rollouts up to 16 policy versions staleHardware2× NVIDIA B200 per harness: vLLM on one, the trainer on the other The training script isexamples/async\_grpo\_harborin TRL.

Below is an animated training curve of the evolution of the number of tool calls on the train set:

For a full write-up, please refer tothe release blogpost.

clem 🤗 (@ClementDelangue): Introducing Qwen3.8-27B-pi-adapted 🤗

We adapted Qwen’s model specifically for the pi coding agent using reinforcement learning and a LoRA adapter.

The results, measured as pass@1: 🚀 SWE-bench Pro (Hard 51): 29.4% → 62.7% 💻 Terminal-Bench 2.1: 39.5% → 55.8% ⚡ 55–62% fewer

Similar Articles

harness-hub/Qwen3.8-27B-pi-adapted

Hugging Face Models Trending

A LoRA adapter for Qwen3.8-27B trained with reinforcement learning to power the pi coding agent, matching the base model's highest-effort thinking accuracy on SWE-bench Pro and Terminal-Bench while generating about 4.5x fewer tokens with thinking off.

Initial experiences with pi adapter on q4 qwen3.8 27b

Reddit r/LocalLLaMA

A developer shares hands-on experience using Hugging Face's pi adapter LoRA on Qwen3.8 27B (Q4_K_S quant) for Penpot design workflows, reporting better tool/harness usage but slower token generation, occasional loop failures, and a lack of ready-made quants making it feel like a research preview rather than a production upgrade.