GLM-5.1vs.Qwen 3.8 27B
Which one is better?
Level in the overall index: the gap is too small for a clear winner. Qwen 3.8 27B is cheaper: GLM-5.1 costs 35 times (input) and 1.8 times (output) as much per token.
11
GLM-5.1
Rank 48 to 55
11
Qwen 3.8 27B
Rank 44 to 57
77
GLM-5.1
fairly slow
40
Qwen 3.8 27B
slow
$2.15
GLM-5.1
fairly expensive
$0.66
Qwen 3.8 27B
fairly cheap
GLM-5.1
- Subscription
No Z.ai subscription includes this model. Through other providers from ≈ €1.06 a month (Command Code Go, with limited credits). 13 plans in Germany.
- Via API
$1.40 input, $4.40 output per 1M tokens according to the provider; the price index has listed it at $1.05 and $3.50 since Sep 3, 2026.
Qwen 3.8 27B
- Subscription
No Alibaba subscription includes this model. Through other providers from ≈ €1.06 a month (Command Code Go, with limited credits). 10 plans in Germany.
- Via API
$0.04 input, $2.50 output per 1M tokens according to the provider; the price index has listed it at $0.42 and $3 since Oct 27, 2026.
Overall index with data through Oct 29, 2026 · Prices from Oct 29, 2026 · Speed: measured by Artificial Analysis · Subscriptions as of Oct 24, 2026 · Subscription prices incl. VAT
Where is the difference?
Values from 0 to 100 per task area. We only name a winner when the gap is large enough and there are enough tests.
GLM-5.1 ahead: 1Qwen 3.8 27B ahead: 1level: 4too little data: 4of 10 areas
AreaGLM-5.1Qwen 3.8 27B0 to 100, higher is betterResult
Areas in the overall index
CodinglevelGLM-5.1: 10Qwen 3.8 27B: 13
Reasoningtoo little dataGLM-5.1: 7Qwen 3.8 27B: 6
Mathematicsonly Qwen 3.8 27B has a scoreGLM-5.1: not enough dataQwen 3.8 27B: 8
KnowledgeGLM-5.1 aheadGLM-5.1: 13Qwen 3.8 27B: 5
AgentslevelGLM-5.1: 17Qwen 3.8 27B: 21
Generalonly GLM-5.1 has a scoreGLM-5.1: 19Qwen 3.8 27B: not enough data
Multimodalonly Qwen 3.8 27B has a scoreGLM-5.1: not enough dataQwen 3.8 27B: 13
Specialist fields
FinanceQwen 3.8 27B aheadGLM-5.1: 13Qwen 3.8 27B: 34
LegallevelGLM-5.1: 19Qwen 3.8 27B: 23
HealthlevelGLM-5.1: 6Qwen 3.8 27B: 3
Show 2 more areas that do not count toward the overall index
AreaGLM-5.1Qwen 3.8 27B0 to 100, higher is betterResult
Cybersecurityonly Qwen 3.8 27B has a scoreGLM-5.1: not enough dataQwen 3.8 27B: 19
Productivityonly GLM-5.1 has a scoreGLM-5.1: 18Qwen 3.8 27B: not enough data
See for yourself: same task, one attempt
SVG short test
One SVG task with the same fixed settings for every model, without hidden thinking.
Shared test round: 43b8c5c46830
GLM-5.1
Test details
API reports 0 reasoning tokensAPI model: z-ai/glm-5.1580 output tokensQwen 3.8 27B
Test details
API reports 0 reasoning tokensAPI model: qwen/qwen3.8-27b865 output tokensEarlier tests
Pelican on a bicycle
A community classic for free-form SVG drawing.
GLM-5.1
Output withheld for safety
Reasoning (requested): high
Current attempt
- Run date
- Oct 5, 2026
- API provider
- openrouter
- Requested API model
- z-ai/glm-5.1
- API model reported in response
- z-ai/glm-5.1
- Reasoning setting
- high
- Pinned endpoint
- Z.AI | z-ai/glm-5.1-20260406
- Temperature
- Provider default
- Token limit including reasoning
- 8,192 tokens
- Test protocol
- community-visual-tests-v1
- Technical checks
- Failed
- Response time
- 141.6 s
Not yet visually reviewed. A successful recording does not confirm correct physics or full compliance with the prompt.
Earlier attempts (3)
- Oct 2, 2026Output withheld for safetyReasoning setting: high
- Oct 1, 2026API request failedReasoning setting: high
- Sep 7, 2026API request failedReasoning setting: high
Qwen 3.8 27B
Not tested yet
Prompt and test conditions
Generate an SVG of a pelican riding a bicycle
Token limit including reasoning: 8,192 tokens
Task origin (Simon Willison)All data side by side
The better value is bold. For the overall index only when the gap is clear.
| Metric | GLM-5.1 | Qwen 3.8 27B |
|---|---|---|
| Intelligence | ||
| Overall index0 to 100, higher is better | 11 | 11 |
| Rank | Rank 48 to 55 of 95 | Rank 44 to 57 of 95 |
| Price per 1M tokens | ||
| Inputwhat you send | $1.40 | $0.04 |
| Outputwhat the model writes | $4.40 | $2.50 |
| Speed | ||
| Tokens per second | 77 | 40 |
| Technical | ||
| Context windowhow much text the model reads at once | 200,000 tokens≈ 300 A4 pages | 262,144 tokens≈ 390 A4 pages |
| Knowledge cutoff | n/a | n/a |
| Released | Apr 7, 2026 | Aug 14, 2026 |
| Availability | Open Source | Active |
| Open weights | yes | yes |
Prices without caching or batch discounts · Page count as a rule of thumb from Artificial Analysis
Shared benchmarks
Single results from the same benchmark version, task and metric. A value is bold only with the same test setup and a clear gap.
A: GLM-5.1B: Qwen 3.8 27B
Code MigrationOverall · higher is betterTest setup not documented as identicalGLM-5.1: 25.8%Qwen 3.8 27B: 14.2%Vals AIAs of 10/18/2026
CritPt v2Overall · higher is betterGLM-5.1: 0.0%Qwen 3.8 27B: 0.3%Artificial AnalysisAs of 10/29/2026
Finance Agent (v2)Overall · higher is betterTest setup not documented as identicalGLM-5.1: 44.8%Qwen 3.8 27B: 48.6%Vals AIAs of 10/20/2026
GPQA DiamondOverall · higher is betterTest setup not documented as identicalGLM-5.1: 84.5%Qwen 3.8 27B: 88.9%Vals AIAs of 09/01/2026
Legal Research BenchOverall · All-pass · higher is betterTest setup not documented as identicalGLM-5.1: 27.9%Qwen 3.8 27B: 36.1%Vals AIAs of 10/26/2026
LegalBenchOverall · higher is betterTest setup not documented as identicalGLM-5.1: 84.4%Qwen 3.8 27B: 82.4%Vals AIAs of 10/13/2026
LiveCodeBenchOverall · higher is betterTest setup not documented as identicalGLM-5.1: 81.4%Qwen 3.8 27B: 84%Vals AIAs of 09/01/2026
MedCodeOverall · higher is betterTest setup not documented as identicalGLM-5.1: 41.6%Qwen 3.8 27B: 28.7%Vals AIAs of 10/13/2026
MedScribeOverall · higher is betterTest setup not documented as identicalGLM-5.1: 72.3%Qwen 3.8 27B: 83.8%Vals AIAs of 10/13/2026
MMLU ProOverall · higher is betterTest setup not documented as identicalGLM-5.1: 86.9%Qwen 3.8 27B: 84.3%Vals AIAs of 09/01/2026
Show 54 more benchmarks
ProgramBenchOverall · higher is betterTest setup not documented as identicalGLM-5.1: 0%Qwen 3.8 27B: 0%Vals AIAs of 10/20/2026
SWE-bench VerifiedOverall · higher is betterTest setup not documented as identicalGLM-5.1: 76.4%Qwen 3.8 27B: 86%Vals AIAs of 09/01/2026
TaxEval v2Overall · higher is betterTest setup not documented as identicalGLM-5.1: 71.2%Qwen 3.8 27B: 70.9%Vals AIAs of 09/01/2026
Terminal-Bench 2.1Overall · higher is betterTest setup not documented as identicalGLM-5.1: 56.9%Qwen 3.8 27B: 58.4%Vals AIAs of 10/01/2026
Vibe Code Bench v1.1Overall · higher is betterTest setup not documented as identicalGLM-5.1: 31.5%Qwen 3.8 27B: 64.9%Vals AIAs of 10/26/2026
Code MigrationCLI · higher is betterTest setup not documented as identicalGLM-5.1: 18.8%Qwen 3.8 27B: 2.2%Vals AIAs of 10/18/2026
Finance Agent (v2)Adjustments · higher is betterTest setup not documented as identicalGLM-5.1: 43.3%Qwen 3.8 27B: 47.3%Vals AIAs of 10/20/2026
GPQA DiamondFew-Shot CoT · higher is betterTest setup not documented as identicalGLM-5.1: 85.8%Qwen 3.8 27B: 88.9%Vals AIAs of 09/01/2026
Legal Research BenchAdministrative / Regulatory · All-pass · higher is betterTest setup not documented as identicalGLM-5.1: 31.8%Qwen 3.8 27B: 45.5%Vals AIAs of 10/26/2026
LegalBenchConclusion Tasks · higher is betterTest setup not documented as identicalGLM-5.1: 89.6%Qwen 3.8 27B: 88.4%Vals AIAs of 10/13/2026
LiveCodeBenchEasy · higher is betterTest setup not documented as identicalGLM-5.1: 98.8%Qwen 3.8 27B: 99.1%Vals AIAs of 09/01/2026
MMLU ProBiology · higher is betterTest setup not documented as identicalGLM-5.1: 93.7%Qwen 3.8 27B: 91.5%Vals AIAs of 09/01/2026
ProgramBenchAlmost Resolved · higher is betterTest setup not documented as identicalGLM-5.1: 1%Qwen 3.8 27B: 1%Vals AIAs of 10/20/2026
SWE-bench Verified<15 min fix · higher is betterTest setup not documented as identicalGLM-5.1: 88.7%Qwen 3.8 27B: 90.2%Vals AIAs of 09/01/2026
TaxEval v2Correctness · higher is betterTest setup not documented as identicalGLM-5.1: 61.5%Qwen 3.8 27B: 60.1%Vals AIAs of 09/01/2026
Terminal-Bench 2.1Easy · higher is betterTest setup not documented as identicalGLM-5.1: 83.3%Qwen 3.8 27B: 50%Vals AIAs of 10/01/2026
Code MigrationCOBOL · higher is betterTest setup not documented as identicalGLM-5.1: 46.6%Qwen 3.8 27B: 50%Vals AIAs of 10/18/2026
Finance Agent (v2)All-Pass · higher is betterTest setup not documented as identicalGLM-5.1: 32.7%Qwen 3.8 27B: 36.8%Vals AIAs of 10/20/2026
GPQA DiamondZero-Shot CoT · higher is betterTest setup not documented as identicalGLM-5.1: 83.2%Qwen 3.8 27B: 88.9%Vals AIAs of 09/01/2026
Legal Research BenchAdministrative / Regulatory · Weighted partial credit · higher is betterTest setup not documented as identicalGLM-5.1: 80%Qwen 3.8 27B: 88.5%Vals AIAs of 10/26/2026
LegalBenchInterpretation Tasks · higher is betterTest setup not documented as identicalGLM-5.1: 81.7%Qwen 3.8 27B: 82%Vals AIAs of 10/13/2026
LiveCodeBenchHard · higher is betterTest setup not documented as identicalGLM-5.1: 57.4%Qwen 3.8 27B: 65.7%Vals AIAs of 09/01/2026
MMLU ProBusiness · higher is betterTest setup not documented as identicalGLM-5.1: 91.1%Qwen 3.8 27B: 90.2%Vals AIAs of 09/01/2026
ProgramBenchFully Resolved · higher is betterTest setup not documented as identicalGLM-5.1: 0%Qwen 3.8 27B: 0%Vals AIAs of 10/20/2026
SWE-bench Verified>4 hours · higher is betterTest setup not documented as identicalGLM-5.1: 33.3%Qwen 3.8 27B: 66.7%Vals AIAs of 09/01/2026
TaxEval v2Stepwise Reasoning · higher is betterTest setup not documented as identicalGLM-5.1: 80.9%Qwen 3.8 27B: 81.6%Vals AIAs of 09/01/2026
Terminal-Bench 2.1Hard · higher is betterTest setup not documented as identicalGLM-5.1: 37.8%Qwen 3.8 27B: 43.3%Vals AIAs of 10/01/2026
Code MigrationCode Quality · higher is betterTest setup not documented as identicalGLM-5.1: 3.544 / 5Qwen 3.8 27B: 4.197 / 5Vals AIAs of 10/18/2026
Finance Agent (v2)Comparables · higher is betterTest setup not documented as identicalGLM-5.1: 41.6%Qwen 3.8 27B: 38.9%Vals AIAs of 10/20/2026
Legal Research BenchBusiness & Commercial · All-pass · higher is betterTest setup not documented as identicalGLM-5.1: 23.3%Qwen 3.8 27B: 41.9%Vals AIAs of 10/26/2026
LegalBenchIssue Tasks · higher is betterTest setup not documented as identicalGLM-5.1: 89%Qwen 3.8 27B: 90.2%Vals AIAs of 10/13/2026
LiveCodeBenchMedium · higher is betterTest setup not documented as identicalGLM-5.1: 88%Qwen 3.8 27B: 87.2%Vals AIAs of 09/01/2026
MMLU ProChemistry · higher is betterTest setup not documented as identicalGLM-5.1: 91.8%Qwen 3.8 27B: 91.8%Vals AIAs of 09/01/2026
ProgramBenchRaw Pass Rate · higher is betterTest setup not documented as identicalGLM-5.1: 50.9%Qwen 3.8 27B: 11.2%Vals AIAs of 10/20/2026
SWE-bench Verified1-4 hours · higher is betterTest setup not documented as identicalGLM-5.1: 45.2%Qwen 3.8 27B: 69%Vals AIAs of 09/01/2026
Terminal-Bench 2.1Medium · higher is betterTest setup not documented as identicalGLM-5.1: 65.5%Qwen 3.8 27B: 67.3%Vals AIAs of 10/01/2026
Finance Agent (v2)Disclosure Analysis · higher is betterTest setup not documented as identicalGLM-5.1: 52.7%Qwen 3.8 27B: 60.3%Vals AIAs of 10/20/2026
Legal Research BenchBusiness & Commercial · Weighted partial credit · higher is betterTest setup not documented as identicalGLM-5.1: 81.7%Qwen 3.8 27B: 86.8%Vals AIAs of 10/26/2026
LegalBenchRhetoric Tasks · higher is betterTest setup not documented as identicalGLM-5.1: 81.1%Qwen 3.8 27B: 81.4%Vals AIAs of 10/13/2026
MMLU ProComputer Science · higher is betterTest setup not documented as identicalGLM-5.1: 89.5%Qwen 3.8 27B: 88.3%Vals AIAs of 09/01/2026
SWE-bench Verified15 min - 1 hour · higher is betterTest setup not documented as identicalGLM-5.1: 72.8%Qwen 3.8 27B: 85.8%Vals AIAs of 09/01/2026
Finance Agent (v2)Earnings Analysis · higher is betterTest setup not documented as identicalGLM-5.1: 57%Qwen 3.8 27B: 70.9%Vals AIAs of 10/20/2026
Legal Research BenchCivil Litigation · All-pass · higher is betterTest setup not documented as identicalGLM-5.1: 22.2%Qwen 3.8 27B: 25.9%Vals AIAs of 10/26/2026
LegalBenchRule Tasks · higher is betterTest setup not documented as identicalGLM-5.1: 80.6%Qwen 3.8 27B: 70.2%Vals AIAs of 10/13/2026
MMLU ProEconomics · higher is betterTest setup not documented as identicalGLM-5.1: 89%Qwen 3.8 27B: 88.2%Vals AIAs of 09/01/2026
Finance Agent (v2)Financial Modeling · higher is betterTest setup not documented as identicalGLM-5.1: 13.5%Qwen 3.8 27B: 19.3%Vals AIAs of 10/20/2026
Legal Research BenchCivil Litigation · Weighted partial credit · higher is betterTest setup not documented as identicalGLM-5.1: 83.5%Qwen 3.8 27B: 83.1%Vals AIAs of 10/26/2026
MMLU ProEngineering · higher is betterTest setup not documented as identicalGLM-5.1: 84%Qwen 3.8 27B: 82.4%Vals AIAs of 09/01/2026
Finance Agent (v2)General Qualitative Analysis · higher is betterTest setup not documented as identicalGLM-5.1: 63.7%Qwen 3.8 27B: 62.1%Vals AIAs of 10/20/2026
Legal Research BenchConstitutional / Civil Rights · All-pass · higher is betterTest setup not documented as identicalGLM-5.1: 34.2%Qwen 3.8 27B: 26.3%Vals AIAs of 10/26/2026
MMLU ProHealth · higher is betterTest setup not documented as identicalGLM-5.1: 81.5%Qwen 3.8 27B: 78.6%Vals AIAs of 09/01/2026
Finance Agent (v2)General Quantitative Analysis · higher is betterTest setup not documented as identicalGLM-5.1: 61.9%Qwen 3.8 27B: 66.9%Vals AIAs of 10/20/2026
Legal Research BenchConstitutional / Civil Rights · Weighted partial credit · higher is betterTest setup not documented as identicalGLM-5.1: 84.9%Qwen 3.8 27B: 82%Vals AIAs of 10/26/2026
MMLU ProHistory · higher is betterTest setup not documented as identicalGLM-5.1: 78.4%Qwen 3.8 27B: 75.6%Vals AIAs of 09/01/2026