Claude Haiku 5.5vs.Qwen 3.8 27B
Which one is better?
Claude Haiku 5.5 is clearly ahead in the overall index. Qwen 3.8 27B is cheaper for input, Claude Haiku 5.5 for output.
57
Claude Haiku 5.5
Rank 16 to 27
11
Qwen 3.8 27B
Rank 44 to 57
241
Claude Haiku 5.5
fast
40
Qwen 3.8 27B
slow
$0.20
Claude Haiku 5.5
cheap
$0.66
Qwen 3.8 27B
fairly cheap
Claude Haiku 5.5
- Subscription
Free to use at Anthropic (Claude Free), more from €21.42 a month (Claude Pro). Through other providers from €6.83 a month (Poe Basic, with limited points). 29 plans in Germany.
- Via API
$0.10 input, $0.50 output per 1M tokens, unchanged since Oct 14, 2026.
Qwen 3.8 27B
- Subscription
No Alibaba subscription includes this model. Through other providers from ≈ €1.06 a month (Command Code Go, with limited credits). 10 plans in Germany.
- Via API
$0.04 input, $2.50 output per 1M tokens according to the provider; the price index has listed it at $0.42 and $3 since Oct 27, 2026.
Overall index with data through Oct 29, 2026 · Prices from Oct 29, 2026 · Speed: measured by Artificial Analysis · Subscriptions as of Oct 24, 2026 · Subscription prices incl. VAT
Where is the difference?
Values from 0 to 100 per task area. We only name a winner when the gap is large enough and there are enough tests.
Haiku 5.5 ahead: 6Qwen 3.8 27B ahead: 0level: 2too little data: 2of 10 areas
AreaHaiku 5.5Qwen 3.8 27B0 to 100, higher is betterResult
Areas in the overall index
CodingHaiku 5.5 aheadHaiku 5.5: 66Qwen 3.8 27B: 13
ReasoningHaiku 5.5 aheadHaiku 5.5: 62Qwen 3.8 27B: 6
MathematicsHaiku 5.5 aheadHaiku 5.5: 69Qwen 3.8 27B: 8
KnowledgelevelHaiku 5.5: 10Qwen 3.8 27B: 5
AgentsHaiku 5.5 aheadHaiku 5.5: 63Qwen 3.8 27B: 21
Generalonly Haiku 5.5 has a scoreHaiku 5.5: 67Qwen 3.8 27B: not enough data
Multimodalonly Qwen 3.8 27B has a scoreHaiku 5.5: not enough dataQwen 3.8 27B: 13
Specialist fields
FinanceHaiku 5.5 aheadHaiku 5.5: 61Qwen 3.8 27B: 34
LegallevelHaiku 5.5: 30Qwen 3.8 27B: 23
HealthHaiku 5.5 aheadHaiku 5.5: 76Qwen 3.8 27B: 3
Show 3 more areas that do not count toward the overall index
AreaHaiku 5.5Qwen 3.8 27B0 to 100, higher is betterResult
CybersecurityHaiku 5.5 aheadHaiku 5.5: 79Qwen 3.8 27B: 19
Productivityonly Haiku 5.5 has a scoreHaiku 5.5: 50Qwen 3.8 27B: not enough data
Educationonly Haiku 5.5 has a scoreHaiku 5.5: 79Qwen 3.8 27B: not enough data
See for yourself: same task, one attempt
SVG short test
One SVG task with the same fixed settings for every model, without hidden thinking.
Shared test round: 43b8c5c46830
Claude Haiku 5.5
Unavailable in this round
Qwen 3.8 27B
Test details
API reports 0 reasoning tokensAPI model: qwen/qwen3.8-27b865 output tokensEarlier tests
This is the lowest documented setting for each provider. It does not make compute comparable across models.
Pelican on a bicycle
A community classic for free-form SVG drawing.
Claude Haiku 5.5
Not tested yet
Qwen 3.8 27B
The response reached the token limit. No complete recording is available.
Reasoning (requested): disabled (enabled: false)
Current attempt
- Run date
- Oct 11, 2026
- API provider
- Alibaba
- Requested API model
- qwen/qwen3.8-27b
- API model reported in response
- qwen/qwen3.8-27b
- Reasoning setting
- disabled (enabled: false)
- Pinned endpoint
- Alibaba | qwen/qwen3.8-27b-20260814
- Temperature
- Provider default
- Token limit including reasoning
- 8,192 tokens
- Reasoning tokens
- 0
- Test protocol
- community-visual-low-v1
- Technical checks
- Failed
- Response time
- 119.1 s
Not yet visually reviewed. A successful recording does not confirm correct physics or full compliance with the prompt.
Prompt and test conditions
Generate an SVG of a pelican riding a bicycle
Token limit including reasoning: 8,192 tokens
Task origin (Simon Willison)All data side by side
The better value is bold. For the overall index only when the gap is clear.
| Metric | Claude Haiku 5.5 | Qwen 3.8 27B |
|---|---|---|
| Intelligence | ||
| Overall index0 to 100, higher is better | 57 | 11 |
| Rank | Rank 16 to 27 of 95 | Rank 44 to 57 of 95 |
| Price per 1M tokens | ||
| Inputwhat you send | $0.10above 100,000 tokens: $0.50 | $0.04 |
| Outputwhat the model writes | $0.50above 100,000 tokens: $2.50 | $2.50 |
| Speed | ||
| Tokens per second | 241 | 40 |
| Technical | ||
| Context windowhow much text the model reads at once | 1 M tokens≈ 1,500 A4 pages | 262,144 tokens≈ 390 A4 pages |
| Knowledge cutoff | June 2026 | n/a |
| Released | Oct 7, 2026 | Aug 14, 2026 |
| Availability | Active | Active |
| Open weights | no | yes |
Prices without caching or batch discounts · Page count as a rule of thumb from Artificial Analysis
Shared benchmarks
Single results from the same benchmark version, task and metric. A value is bold only with the same test setup and a clear gap.
A: Claude Haiku 5.5B: Qwen 3.8 27B
AA-BriefcaseOverall · higher is betterClaude Haiku 5.5: 1,524.29 eloQwen 3.8 27B: 1,392.67 eloArtificial AnalysisAs of 10/29/2026
AA-OmniscienceOverall · higher is betterClaude Haiku 5.5: 7.833 pointsQwen 3.8 27B: -6.367 pointsArtificial AnalysisAs of 10/29/2026
AA-Omniscience AccuracyOverall · higher is betterClaude Haiku 5.5: 33.8%Qwen 3.8 27B: 17.3%Artificial AnalysisAs of 10/29/2026
AA-Omniscience Hallucination RateOverall · lower is betterClaude Haiku 5.5: 39.2%Qwen 3.8 27B: 28.6%Artificial AnalysisAs of 10/29/2026
Artificial Analysis Intelligence IndexOverall · higher is betterClaude Haiku 5.5: 34.339 pointsQwen 3.8 27B: 20.107 pointsArtificial AnalysisAs of 10/29/2026
AutomationBench-AAOverall · higher is betterClaude Haiku 5.5: 36.0%Qwen 3.8 27B: 48.2%Artificial AnalysisAs of 10/29/2026
Code MigrationOverall · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 44.6%Qwen 3.8 27B: 14.2%Vals AIAs of 10/18/2026
CritPt v2Overall · higher is betterClaude Haiku 5.5: 22.6%Qwen 3.8 27B: 5.4%Artificial AnalysisAs of 10/29/2026
CritPt v2 (Hard Subset)Overall · higher is betterClaude Haiku 5.5: 44.5%Qwen 3.8 27B: 4.0%Artificial AnalysisAs of 10/29/2026
CyberBench v1.1Overall · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 75.5%Qwen 3.8 27B: 67.8%Vals AIAs of 10/20/2026
Show 54 more benchmarks
EMBOverall · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 59%Qwen 3.8 27B: 59.7%Vals AIAs of 10/20/2026
Finance Agent (v2)Overall · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 54.1%Qwen 3.8 27B: 48.6%Vals AIAs of 10/20/2026
FrontierCodeOverall · higher is betterClaude Haiku 5.5: 29.9%Qwen 3.8 27B: 14.9%Artificial AnalysisAs of 10/29/2026
FrontierCode 1.1 MainOverall · higher is betterClaude Haiku 5.5: 45.5%Qwen 3.8 27B: 34.2%Artificial AnalysisAs of 10/29/2026
GDP.pdfOverall · higher is betterClaude Haiku 5.5: 18.2%Qwen 3.8 27B: 16.4%Artificial AnalysisAs of 10/29/2026
GDPval-AAOverall · higher is betterClaude Haiku 5.5: 1,520.97 eloQwen 3.8 27B: 1,427.26 eloArtificial AnalysisAs of 10/29/2026
HLE-DiamondOverall · higher is betterClaude Haiku 5.5: 28.8%Qwen 3.8 27B: 15.3%Artificial AnalysisAs of 10/29/2026
IOIOverall · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 47.3%Qwen 3.8 27B: 39.1%Vals AIAs of 10/24/2026
Legal Research BenchOverall · All-pass · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 43.8%Qwen 3.8 27B: 36.1%Vals AIAs of 10/26/2026
MedCodeOverall · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 51.5%Qwen 3.8 27B: 28.7%Vals AIAs of 10/13/2026
MedScribeOverall · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 89.5%Qwen 3.8 27B: 83.8%Vals AIAs of 10/13/2026
ProgramBenchOverall · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 2.5%Qwen 3.8 27B: 0%Vals AIAs of 10/20/2026
ProofBench v1.1Overall · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 89%Qwen 3.8 27B: 16%Vals AIAs of 10/24/2026
Tax Agent BenchOverall · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 31.9%Qwen 3.8 27B: 30.9%Vals AIAs of 10/26/2026
Terminal-BenchOverall · higher is betterClaude Haiku 5.5: 29.3%Qwen 3.8 27B: 5.6%Artificial AnalysisAs of 10/29/2026
Terminal-Bench-ScienceOverall · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 10%Qwen 3.8 27B: 0%Vals AIAs of 10/25/2026
Terminal-Bench-ScienceOverall · higher is betterClaude Haiku 5.5: 18.1%Qwen 3.8 27B: 0.5%Artificial AnalysisAs of 10/29/2026
Vibe Code Bench v1.1Overall · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 90.4%Qwen 3.8 27B: 64.9%Vals AIAs of 10/26/2026
AA-Omniscience AccuracyOverall · higher is betterClaude Haiku 5.5: 30.4%Qwen 3.8 27B: 18.3%Artificial AnalysisAs of 10/29/2026
AA-Omniscience Hallucination RateOverall · lower is betterClaude Haiku 5.5: 40.5%Qwen 3.8 27B: 48.7%Artificial AnalysisAs of 10/29/2026
Code MigrationCLI · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 36.1%Qwen 3.8 27B: 2.2%Vals AIAs of 10/18/2026
CritPt v2Overall · higher is betterClaude Haiku 5.5: 9.1%Qwen 3.8 27B: 0.0%Artificial AnalysisAs of 10/29/2026
CyberBench v1.1Patch · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 89.3%Qwen 3.8 27B: 83.9%Vals AIAs of 10/20/2026
EMBComps · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 77.9%Qwen 3.8 27B: 53.3%Vals AIAs of 10/20/2026
Finance Agent (v2)Adjustments · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 49.6%Qwen 3.8 27B: 47.3%Vals AIAs of 10/20/2026
FrontierCode 1.1 MainOverall · higher is betterClaude Haiku 5.5: 29.3%Qwen 3.8 27B: 28.9%Artificial AnalysisAs of 10/29/2026
GDPval-AAOverall · higher is betterClaude Haiku 5.5: 1,130.82 eloQwen 3.8 27B: 1,370.88 eloArtificial AnalysisAs of 10/29/2026
IOIIOI 2024 · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 67.7%Qwen 3.8 27B: 31.8%Vals AIAs of 10/24/2026
Legal Research BenchAdministrative / Regulatory · All-pass · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 50%Qwen 3.8 27B: 45.5%Vals AIAs of 10/26/2026
ProgramBenchAlmost Resolved · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 29.5%Qwen 3.8 27B: 1%Vals AIAs of 10/20/2026
Tax Agent BenchControversy & Precedence Analysis · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 6.7%Qwen 3.8 27B: 6.7%Vals AIAs of 10/26/2026
Terminal-Bench-ScienceEarth Sciences · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 0%Qwen 3.8 27B: 0%Vals AIAs of 10/25/2026
AA-Omniscience AccuracyOverall · higher is betterClaude Haiku 5.5: 31.7%Qwen 3.8 27B: 17.9%Artificial AnalysisAs of 10/29/2026
AA-Omniscience Hallucination RateOverall · lower is betterClaude Haiku 5.5: 40.4%Qwen 3.8 27B: 65.4%Artificial AnalysisAs of 10/29/2026
Code MigrationCOBOL · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 70%Qwen 3.8 27B: 50%Vals AIAs of 10/18/2026
CritPt v2Overall · higher is betterClaude Haiku 5.5: 12.9%Qwen 3.8 27B: 0.0%Artificial AnalysisAs of 10/29/2026
CyberBench v1.1PoC · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 61.7%Qwen 3.8 27B: 51.7%Vals AIAs of 10/20/2026
EMBDataroom · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 85.5%Qwen 3.8 27B: 79.7%Vals AIAs of 10/20/2026
Finance Agent (v2)All-Pass · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 42.9%Qwen 3.8 27B: 36.8%Vals AIAs of 10/20/2026
FrontierCode 1.1 MainOverall · higher is betterClaude Haiku 5.5: 40.4%Qwen 3.8 27B: 34.7%Artificial AnalysisAs of 10/29/2026
GDPval-AAOverall · higher is betterClaude Haiku 5.5: 1,283 eloQwen 3.8 27B: 1,389.29 eloArtificial AnalysisAs of 10/29/2026
IOIIOI 2025 · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 31.3%Qwen 3.8 27B: 48.5%Vals AIAs of 10/24/2026
Legal Research BenchAdministrative / Regulatory · Weighted partial credit · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 88.7%Qwen 3.8 27B: 88.5%Vals AIAs of 10/26/2026
ProgramBenchFully Resolved · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 2.5%Qwen 3.8 27B: 0%Vals AIAs of 10/20/2026
Tax Agent BenchControversy & Precedence Analysis · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 54.1%Qwen 3.8 27B: 48.3%Vals AIAs of 10/26/2026
Terminal-Bench-ScienceEngineering Sciences · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 0%Qwen 3.8 27B: 0%Vals AIAs of 10/25/2026
Code MigrationCode Quality · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 4.825 / 5Qwen 3.8 27B: 4.197 / 5Vals AIAs of 10/18/2026
EMBDCF · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 55.1%Qwen 3.8 27B: 50%Vals AIAs of 10/20/2026
Finance Agent (v2)Comparables · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 45.3%Qwen 3.8 27B: 38.9%Vals AIAs of 10/20/2026
IOIIOI 2026 · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 42.8%Qwen 3.8 27B: 36.8%Vals AIAs of 10/24/2026
Legal Research BenchBusiness & Commercial · All-pass · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 44.2%Qwen 3.8 27B: 41.9%Vals AIAs of 10/26/2026
ProgramBenchRaw Pass Rate · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 78.7%Qwen 3.8 27B: 11.2%Vals AIAs of 10/20/2026
Tax Agent BenchCurrent & Temporal Analysis · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 36.7%Qwen 3.8 27B: 43.1%Vals AIAs of 10/26/2026
Terminal-Bench-ScienceLife Sciences · higher is betterTest setup not documented as identicalClaude Haiku 5.5: 15.8%Qwen 3.8 27B: 0%Vals AIAs of 10/25/2026
Compare another matchup
More matchups
Related LLM comparisons
Compare Claude Haiku 5.5 and Qwen 3.8 27B with other leading models.