Five Small Qwen Models Just Matched Claude at Coding — With No Extra Training
Key Takeaways
- GVS5H five Qwen vs Claude coding is the headline result of late October 2026: five fresh instances of Alibaba’s open-weight Qwen3.8-27B, coordinated through shared files, scored 92.4% pass@1 on the 100 hardest recent LiveCodeBench problems, against 90.4% for Claude Fable 5 called once.
- The viral “9% of the cost” claim is wrong for the 27B model. That figure belongs to the larger Qwen3.8-Flash-Next. At list prices, the orchestrated 27B costs about 78% of Fable 5, not 9%.
- The 2-point lead is not statistically significant. The honest reading is a tie, not a win, and the result has not yet been independently reproduced.
- No training was involved. The gain came entirely from orchestration: a manager, fresh workers, a verifier that runs public tests, and a shared file-based ledger.
- The comparison is uneven by design. The Qwen side gets up to 10 manager-to-worker rounds per problem plus test feedback, while Fable 5 gets a single call.
A Small Team in a Trenchcoat
On 25 October 2026, coverage of a GitHub project called GVS5H, short for “Gorilla vs. 5 Humans,” spread through AI news roundups. The name jokes about an old internet debate, but the idea is serious: instead of asking one model to solve a hard coding problem in a single pass, give the same model five fresh brains and a shared notebook, then let it manage itself. That is the GVS5H five Qwen vs Claude coding setup in a nutshell.
The researchers behind the project are associated with Persis Capital, and the method needs no extra training data, no fine-tuning, and no task-specific demonstrations. Nothing about the model changed. What changed was how it was organized at inference time, and that alone lifted Qwen3.8-27B from 66.8% on its own to 92.4% under orchestration, a gain of 25.6 percentage points.
But GVS5H deserves a closer look than the viral version got: it mixed up which model the cost claim belonged to, and it overstated what a 2-point gap on 100 problems proves. Sourcing note: the numbers here are reported claims from the project and from write-ups of its paper tables, not results we have independently reproduced.
How the GVS5H Harness Works
The harness works like a tiny software team, and its key design choice is that team members never talk to each other directly. Every instance starts with an empty context and reads state only from files on a shared disk, which makes the system legible, debuggable, and cheap to restart.
A manager instance reads the problem and writes a short strategy plus three to six seed tasks into a plan file. A worker brainstorms the core difficulty and candidate approaches. Then fresh workers execute one task at a time, each rewriting the current solution and a curated set of notes. A verifier runs the candidate against the problem’s public tests, and a failing test overrides any claim of completion. A finalizer writes up the definitive solution when the round budget runs out or the manager declares the work done.
The workspace holds five files: task.md with the problem statement, plan.md with the manager’s plan (capped at 4,000 characters), notes.md with curated findings (capped at 8,000 characters and rewritten rather than appended to), solution.py with the current solution, and tasks.json as a task-list mirror. Each worker sees only the ledger plus its single assigned task. There is no conversation history. There is only the workspace.
The caps matter. Unbounded notes are how agent loops rot: each worker inherits a longer, messier context than the last, and early wrong ideas get carried forward forever. Forcing workers to rewrite the notes file keeps the ledger short and forces each worker to decide what is worth keeping. The loop repeats until the manager says done, reissues the same task, or 10 rounds pass, with low temperatures throughout: 0.3 for the plan, 0.4 for the brainstorm, and 0.2 for execution. Fresh contexts also fix a single-call weakness: at the 128k cap, 150 of 500 single calls (30%) hit the cap and 35 (7%) produced no code at all, while at the full 250k the single call scores 70.0, not 66.8.
The Benchmark: Five Qwen vs Claude on 100 Hard Problems
The benchmark is LCB-100: the 100 most recent problems in the hard split of LiveCodeBench release_v6, chosen by contest date to limit training-data contamination. Each model ran with reasoning on, a 128k output cap, and five passes per arm. The headline metric is pass@1, the share of problems solved on the first attempt within a pass.
| Model | Single call | With manager | Gain | vs Fable 5 |
|---|---|---|---|---|
| Claude Fable 5 | 90.4 ± 1.4 | not run | — | reference |
| Qwen3.8-27B (local, FP8, vLLM) | 66.8 ± 5.2 | 92.4 ± 2.3 | +25.6 | +2.0 |
| Qwen3.8-Flash-Next (local, FP8) | 84.2 ± 1.8 | 93.0 ± 1.2 | +8.8 | +2.6 |
| DeepSeek-V4.1-Flash | 88.4 ± 2.6 | 92.0 ± 1.8 | +3.6 | +1.6 |
| GPT-5.6-Terra | 80.8 ± 1.0 | 88.0 ± 1.2 | +7.2 | −2.4 |
| GPT-5.6-Luna | 70.4 ± 4.6 | 81.2 ± 2.8 | +10.8 | −9.2 |
Two patterns stand out. All five models that ran both arms gained from the manager, and the weaker the single call, the bigger the gain: the smaller the model, the more the scaffold has to add. And the larger open sibling did even better than the 27B, with Qwen3.8-Flash-Next reaching 93.0% under orchestration, the highest score in the table.
Limits worth stating: LiveCodeBench is a 2025-era test some researchers consider saturated, a model released after those contests could still have seen the problems in training, and the project patched two evaluator issues itself (non-unique reference answers on four problems, a standard-input mock bug). Nobody independent has reproduced the results yet.
The Correction: What the Viral “9% of the Cost” Claim Gets Wrong
This is the part of the story most reposts got wrong, and the correction changes the conclusion. The project’s abstract says: “Orchestrated Qwen3.8 Flash Next scores 93.0% against Fable 5’s 90.4% at 9% of the cost, while the smaller Qwen3.8-27B reaches 92.4%.” The 9% figure applies to Flash-Next only. Attaching it to the 27B, as viral retellings did, is simply incorrect.
The project’s cost table, priced at list rates per pass over all 100 problems, makes the error plain:
| Arm | Cost per pass | Cost per solved problem |
|---|---|---|
| Fable 5, single call | $61.11 | $0.68 |
| Qwen3.8-27B, single call | $18.95 | $0.28 |
| Qwen3.8-27B, with manager | $47.82 | $0.52 |
| Qwen3.8-Flash-Next, with manager | $5.76 | $0.062 |
| GPT-5.6-Luna, with manager | $1.50 | $0.018 |
Flash-Next at $5.76 against Fable 5 at $61.11 is about 9%. The 27B with the manager at $47.82 is roughly 78% of Fable 5. The paper itself calls that “a slightly lower price,” which is fair. It is a saving, but not the order-of-magnitude saving the viral claim suggested.
Why is the 27B so much more expensive than its sibling under orchestration? Because it talks a lot while it reasons. Its single call alone emits 7.42 million output tokens per pass, against 1.20 million for Fable 5, and the manager arm pushes that to about 18.6 million output tokens per pass. Those dollar figures are OpenRouter list rates: $0.21 per million input tokens and $2.55 per million output tokens for the 27B. On your own GPU the marginal cost is electricity and waiting time, but the waiting time is not trivial: one community member reported about 50 tokens per second for Qwen3.8-27B on a consumer RX 7900 XTX card, putting a single manager-arm pass at roughly 100 hours of generation for 100 problems on one card, assuming no batching.
Does Qwen3.8-27B Actually Beat Fable 5? Statistically, No
The 2-point gap does not hold up, and to the project’s credit, its own paper says so. The p-value for Qwen3.8-27B with the manager versus Fable 5 single is 0.88 after Holm correction, and the one-sided 95% lower bound on the difference is −0.8 points. In plain language: you can say the 27B matches Fable 5. You cannot say it beats it.
Further caution is warranted. Fable 5 was run single-only due to its high cost, so nobody knows what Fable 5 with a manager would score. The manager arm gets up to 10 rounds of calls per problem plus test feedback, while Fable 5 gets one call and none. The verifier step covers 73 of the 100 problems, and a fairer baseline would have been a single call with one test-and-repair round. Fable 5 also returned a refusal on 6 of 500 problem-passes, costing it between 0 and 1.2 points.
None of this makes the result uninteresting. A tie under favorable conditions for the smaller model is still a remarkable outcome for a 27-billion-parameter open model against a frontier system. But a tie under favorable conditions is a different headline from a win, and headlines shape budgets.
The gains are also not universal, the most useful warning for anyone planning to copy the method. On the project’s OpenRouter-served set, Qwen3.6-35B with reasoning off fell from 35% to 27% at 128k under orchestration, and Kimi-K3 with reasoning on stayed flat at 86%. Orchestration can make a model worse, not just better, by amplifying an early wrong idea through every subsequent worker.
What It Means for Developers
Strip away the benchmark drama, and GVS5H leaves a practical pattern any developer building a coding agent can borrow today. The pattern is not “use five models.” It is “use one model, five times, with a ledger.”
Cap the notes file and make workers rewrite it instead of appending. Unbounded context is how agent loops rot. A rewrite forces selection, and selection is where quality comes from.
Give each worker one task plus the ledger, never the full transcript. A worker that sees only what it needs cannot be anchored by an early wrong turn, and it costs fewer tokens per call.
Treat a failing public test as ground truth that overrides the manager’s “done.” The verifier is the most powerful role in the harness, because it replaces opinion with execution. If you build one piece of this system, build the test runner.
Add a no-progress guard. If the manager hands out the same task twice, stop. Looping on a stuck task burns tokens and time without improving the solution.
Occasionally spawn a worker that sees only the problem statement. The paper itself suggests this, because inherited notes anchor every worker to an early idea. A fresh-eyes worker is cheap insurance against a confident wrong plan.
The strategic takeaway goes beyond code: the next leap in coding performance may come from how models are organized at inference time, not only from how big they are trained to be. A 27B model you can run yourself, given a decent scaffold and a test runner, reached frontier-level pass@1 on hard problems. What it did not do is beat a frontier model outright, and what it did not do is cost 9% as much.
If you run a 27B locally, GVS5H is worth trying, with eyes open about the time cost and the uneven comparison. And if you see the next headline claiming a small model beat a big one, ask three questions first: how many calls did each side get, did either side get to run tests, and is the gap bigger than the error bars. Those three questions would have caught the 9% error before it traveled around the world.
Sources and Further Reading
- The GVS5H project on GitHub, including the paper source tables cited above.
- Authon.dev, “Local Qwen3.8-27B vs Claude Fable 5: It Ties at Coding, but Costs 78%, Not 9%” (25 October 2026), the read of the paper tables behind the cost correction here.
- Paktechstan, “Five Small Open Models Just Outscored Claude’s Best Coder — With No Extra Training” (25 October 2026), the original news coverage of the result.
- ai0.news daily digest, 25 October 2026, which first surfaced the result.
- For context on the open-model wave this result belongs to, see our coverage of Kimi K3.1 Pro: Moonshot AI’s 3-Trillion-Parameter Open-Weight Model and Abacus AI Smaug: Three Open Models, Real Benchmarks, and the Personal-AI Shift.
