GPT-6.1 Solvs.Qwen 3.8 27B
Which one is better?
GPT-6.1 Sol is clearly ahead in the overall index, but costs 50 times (input) and 4 times (output) as much per token.
90
GPT-6.1 Sol
Rank 6 to 9
11
Qwen 3.8 27B
Rank 44 to 57
79
GPT-6.1 Sol
fairly slow
40
Qwen 3.8 27B
slow
$4
GPT-6.1 Sol
fairly expensive
$0.66
Qwen 3.8 27B
fairly cheap
GPT-6.1 Sol
- Subscription
At OpenAI from €23 a month (ChatGPT Plus). Through other providers from ≈ €6.36 a month (ElevenLabs Starter, with limited credits). 58 plans in Germany.
- Via API
$2 input, $10 output per 1M tokens.
Qwen 3.8 27B
- Subscription
No Alibaba subscription includes this model. Through other providers from ≈ €1.06 a month (Command Code Go, with limited credits). 10 plans in Germany.
- Via API
$0.04 input, $2.50 output per 1M tokens according to the provider; the price index has listed it at $0.42 and $3 since Oct 27, 2026.
Overall index with data through Oct 29, 2026 · Prices from Oct 29, 2026 · Speed: measured by Artificial Analysis · Subscriptions as of Oct 24, 2026 · Subscription prices incl. VAT
Where is the difference?
Values from 0 to 100 per task area. We only name a winner when the gap is large enough and there are enough tests.
GPT-6.1 Sol ahead: 9Qwen 3.8 27B ahead: 0level: 0too little data: 1of 10 areas
AreaGPT-6.1 SolQwen 3.8 27B0 to 100, higher is betterResult
Areas in the overall index
CodingGPT-6.1 Sol aheadGPT-6.1 Sol: 91Qwen 3.8 27B: 13
ReasoningGPT-6.1 Sol aheadGPT-6.1 Sol: 95Qwen 3.8 27B: 6
MathematicsGPT-6.1 Sol aheadGPT-6.1 Sol: 95Qwen 3.8 27B: 8
KnowledgeGPT-6.1 Sol aheadGPT-6.1 Sol: 95Qwen 3.8 27B: 5
AgentsGPT-6.1 Sol aheadGPT-6.1 Sol: 77Qwen 3.8 27B: 21
Generalonly GPT-6.1 Sol has a scoreGPT-6.1 Sol: 94Qwen 3.8 27B: not enough data
MultimodalGPT-6.1 Sol aheadGPT-6.1 Sol: 88Qwen 3.8 27B: 13
Specialist fields
FinanceGPT-6.1 Sol aheadGPT-6.1 Sol: 65Qwen 3.8 27B: 34
LegalGPT-6.1 Sol aheadGPT-6.1 Sol: 68Qwen 3.8 27B: 23
HealthGPT-6.1 Sol aheadGPT-6.1 Sol: 75Qwen 3.8 27B: 3
Show 4 more areas that do not count toward the overall index
AreaGPT-6.1 SolQwen 3.8 27B0 to 100, higher is betterResult
Cybersecuritytoo little dataGPT-6.1 Sol: 47Qwen 3.8 27B: 19
Searchonly GPT-6.1 Sol has a scoreGPT-6.1 Sol: 92Qwen 3.8 27B: not enough data
Productivityonly GPT-6.1 Sol has a scoreGPT-6.1 Sol: 51Qwen 3.8 27B: not enough data
Educationonly GPT-6.1 Sol has a scoreGPT-6.1 Sol: 92Qwen 3.8 27B: not enough data
See for yourself: same task, one attempt
The same prompt for both models. We show the latest attempt without fixes; it does not count toward the overall index.
Pelican on a bicycle
A community classic for free-form SVG drawing.
GPT-6.1 Sol
Reasoning (requested): high
Current attempt
- Run date
- Oct 1, 2026
- API provider
- openrouter
- Requested API model
- openai/gpt-6.1-sol
- API model reported in response
- openai/gpt-6.1-sol
- Reasoning setting
- high
- Pinned endpoint
- OpenAI | openai/gpt-6.1-sol-20260929
- Temperature
- Provider default
- Token limit including reasoning
- 8,192 tokens
- Test protocol
- community-visual-tests-v1
- Technical checks
- Passed, not a quality rating
- Response time
- 83.6 s
Visually reviewed. This individual test does not establish overall model quality.
Qwen 3.8 27B
Not tested yet
Prompt and test conditions
Generate an SVG of a pelican riding a bicycle
Token limit including reasoning: 8,192 tokens
Task origin (Simon Willison)SVG short test
SVG short test
One SVG task with the same fixed settings for every model, without hidden thinking.
Shared test round: 43b8c5c46830
GPT-6.1 Sol
Unavailable in this round
Qwen 3.8 27B
Test details
API reports 0 reasoning tokensAPI model: qwen/qwen3.8-27b865 output tokensAll data side by side
The better value is bold. For the overall index only when the gap is clear.
| Metric | GPT-6.1 Sol | Qwen 3.8 27B |
|---|---|---|
| Intelligence | ||
| Overall index0 to 100, higher is better | 90 | 11 |
| Rank | Rank 6 to 9 of 95 | Rank 44 to 57 of 95 |
| Price per 1M tokens | ||
| Inputwhat you send | $2above 272,000 tokens: $4 | $0.04 |
| Outputwhat the model writes | $10above 272,000 tokens: $15 | $2.50 |
| Speed | ||
| Tokens per second | 79 | 40 |
| Technical | ||
| Context windowhow much text the model reads at once | 1.05 M tokens≈ 1,580 A4 pages | 262,144 tokens≈ 390 A4 pages |
| Knowledge cutoff | Apr 30, 2026 | n/a |
| Released | Sep 29, 2026 | Aug 14, 2026 |
| Availability | Active | Active |
| Open weights | no | yes |
Prices without caching or batch discounts · Page count as a rule of thumb from Artificial Analysis
Shared benchmarks
Single results from the same benchmark version, task and metric. A value is bold only with the same test setup and a clear gap.
A: GPT-6.1 SolB: Qwen 3.8 27B
AA-BriefcaseOverall · higher is betterGPT-6.1 Sol: 1,501.7 eloQwen 3.8 27B: 1,392.67 eloArtificial AnalysisAs of 10/29/2026
AA-OmniscienceOverall · higher is betterGPT-6.1 Sol: 42.367 pointsQwen 3.8 27B: -6.367 pointsArtificial AnalysisAs of 10/29/2026
AA-Omniscience AccuracyOverall · higher is betterGPT-6.1 Sol: 61.1%Qwen 3.8 27B: 17.3%Artificial AnalysisAs of 10/29/2026
AA-Omniscience Hallucination RateOverall · lower is betterGPT-6.1 Sol: 48.0%Qwen 3.8 27B: 28.6%Artificial AnalysisAs of 10/29/2026
Artificial Analysis Intelligence IndexOverall · higher is betterGPT-6.1 Sol: 52.308 pointsQwen 3.8 27B: 20.107 pointsArtificial AnalysisAs of 10/29/2026
AutomationBench-AAOverall · higher is betterGPT-6.1 Sol: 66.6%Qwen 3.8 27B: 48.2%Artificial AnalysisAs of 10/29/2026
Code MigrationOverall · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 65.1%Qwen 3.8 27B: 14.2%Vals AIAs of 10/18/2026
CritPt v2Overall · higher is betterGPT-6.1 Sol: 31.7%Qwen 3.8 27B: 5.4%Artificial AnalysisAs of 10/29/2026
CritPt v2 (Hard Subset)Overall · higher is betterGPT-6.1 Sol: 68.0%Qwen 3.8 27B: 4.0%Artificial AnalysisAs of 10/29/2026
CyberBench v1.1Overall · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 39.3%Qwen 3.8 27B: 67.8%Vals AIAs of 10/20/2026
Show 54 more benchmarks
EMBOverall · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 70.8%Qwen 3.8 27B: 59.7%Vals AIAs of 10/20/2026
Finance Agent (v2)Overall · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 52%Qwen 3.8 27B: 48.6%Vals AIAs of 10/20/2026
FrontierCodeOverall · higher is betterGPT-6.1 Sol: 38.5%Qwen 3.8 27B: 14.9%Artificial AnalysisAs of 10/29/2026
FrontierCode 1.1 MainOverall · higher is betterGPT-6.1 Sol: 46.3%Qwen 3.8 27B: 34.2%Artificial AnalysisAs of 10/29/2026
GDP.pdfOverall · higher is betterGPT-6.1 Sol: 31.8%Qwen 3.8 27B: 16.4%Artificial AnalysisAs of 10/29/2026
GDPval-AAOverall · higher is betterGPT-6.1 Sol: 1,552.61 eloQwen 3.8 27B: 1,427.26 eloArtificial AnalysisAs of 10/29/2026
HLE-DiamondOverall · higher is betterGPT-6.1 Sol: 55.8%Qwen 3.8 27B: 15.3%Artificial AnalysisAs of 10/29/2026
IOIOverall · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 96.9%Qwen 3.8 27B: 39.1%Vals AIAs of 10/24/2026
Legal Research BenchOverall · All-pass · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 38.5%Qwen 3.8 27B: 36.1%Vals AIAs of 10/26/2026
MedCodeOverall · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 48.8%Qwen 3.8 27B: 28.7%Vals AIAs of 10/13/2026
MedScribeOverall · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 86.5%Qwen 3.8 27B: 83.8%Vals AIAs of 10/13/2026
MMMU-ProOverall · higher is betterGPT-6.1 Sol: 85.1%Qwen 3.8 27B: 76.3%Artificial AnalysisAs of 10/29/2026
ProgramBenchOverall · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 3%Qwen 3.8 27B: 0%Vals AIAs of 10/20/2026
ProofBench v1.1Overall · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 99%Qwen 3.8 27B: 16%Vals AIAs of 10/24/2026
Tax Agent BenchOverall · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 21.8%Qwen 3.8 27B: 30.9%Vals AIAs of 10/26/2026
Terminal-BenchOverall · higher is betterGPT-6.1 Sol: 54.0%Qwen 3.8 27B: 5.6%Artificial AnalysisAs of 10/29/2026
Terminal-Bench-ScienceOverall · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 52.9%Qwen 3.8 27B: 0%Vals AIAs of 10/25/2026
Terminal-Bench-ScienceOverall · higher is betterGPT-6.1 Sol: 50.5%Qwen 3.8 27B: 0.5%Artificial AnalysisAs of 10/29/2026
Vibe Code Bench v1.1Overall · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 88.9%Qwen 3.8 27B: 64.9%Vals AIAs of 10/26/2026
AA-Omniscience AccuracyOverall · higher is betterGPT-6.1 Sol: 59.6%Qwen 3.8 27B: 17.9%Artificial AnalysisAs of 10/29/2026
AA-Omniscience Hallucination RateOverall · lower is betterGPT-6.1 Sol: 48.1%Qwen 3.8 27B: 65.4%Artificial AnalysisAs of 10/29/2026
Code MigrationCLI · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 63.5%Qwen 3.8 27B: 2.2%Vals AIAs of 10/18/2026
CritPt v2Overall · higher is betterGPT-6.1 Sol: 27.7%Qwen 3.8 27B: 0.0%Artificial AnalysisAs of 10/29/2026
CyberBench v1.1Patch · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 78.6%Qwen 3.8 27B: 83.9%Vals AIAs of 10/20/2026
EMBComps · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 61.8%Qwen 3.8 27B: 53.3%Vals AIAs of 10/20/2026
Finance Agent (v2)Adjustments · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 44.7%Qwen 3.8 27B: 47.3%Vals AIAs of 10/20/2026
FrontierCode 1.1 MainOverall · higher is betterGPT-6.1 Sol: 43.3%Qwen 3.8 27B: 34.7%Artificial AnalysisAs of 10/29/2026
GDPval-AAOverall · higher is betterGPT-6.1 Sol: 1,459.4 eloQwen 3.8 27B: 1,389.29 eloArtificial AnalysisAs of 10/29/2026
IOIIOI 2024 · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 100%Qwen 3.8 27B: 31.8%Vals AIAs of 10/24/2026
Legal Research BenchAdministrative / Regulatory · All-pass · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 53%Qwen 3.8 27B: 45.5%Vals AIAs of 10/26/2026
MMMU-ProOverall · higher is betterGPT-6.1 Sol: 83.9%Qwen 3.8 27B: 74.2%Artificial AnalysisAs of 10/29/2026
ProgramBenchAlmost Resolved · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 46%Qwen 3.8 27B: 1%Vals AIAs of 10/20/2026
Tax Agent BenchControversy & Precedence Analysis · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 6.7%Qwen 3.8 27B: 6.7%Vals AIAs of 10/26/2026
Terminal-Bench-ScienceEarth Sciences · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 62.5%Qwen 3.8 27B: 0%Vals AIAs of 10/25/2026
AA-Omniscience AccuracyOverall · higher is betterGPT-6.1 Sol: 58.6%Qwen 3.8 27B: 18.3%Artificial AnalysisAs of 10/29/2026
AA-Omniscience Hallucination RateOverall · lower is betterGPT-6.1 Sol: 49.9%Qwen 3.8 27B: 48.7%Artificial AnalysisAs of 10/29/2026
Code MigrationCOBOL · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 70%Qwen 3.8 27B: 50%Vals AIAs of 10/18/2026
CritPt v2Overall · higher is betterGPT-6.1 Sol: 24.9%Qwen 3.8 27B: 0.0%Artificial AnalysisAs of 10/29/2026
CyberBench v1.1PoC · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 0%Qwen 3.8 27B: 51.7%Vals AIAs of 10/20/2026
EMBDataroom · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 84.5%Qwen 3.8 27B: 79.7%Vals AIAs of 10/20/2026
Finance Agent (v2)All-Pass · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 39.9%Qwen 3.8 27B: 36.8%Vals AIAs of 10/20/2026
FrontierCode 1.1 MainOverall · higher is betterGPT-6.1 Sol: 36.7%Qwen 3.8 27B: 28.9%Artificial AnalysisAs of 10/29/2026
GDPval-AAOverall · higher is betterGPT-6.1 Sol: 1,322.56 eloQwen 3.8 27B: 1,370.88 eloArtificial AnalysisAs of 10/29/2026
IOIIOI 2025 · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 100%Qwen 3.8 27B: 48.5%Vals AIAs of 10/24/2026
Legal Research BenchAdministrative / Regulatory · Weighted partial credit · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 85.5%Qwen 3.8 27B: 88.5%Vals AIAs of 10/26/2026
MMMU-ProOverall · higher is betterGPT-6.1 Sol: 83.1%Qwen 3.8 27B: 73.8%Artificial AnalysisAs of 10/29/2026
ProgramBenchFully Resolved · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 3%Qwen 3.8 27B: 0%Vals AIAs of 10/20/2026
Tax Agent BenchControversy & Precedence Analysis · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 64.8%Qwen 3.8 27B: 48.3%Vals AIAs of 10/26/2026
Terminal-Bench-ScienceEngineering Sciences · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 33.3%Qwen 3.8 27B: 0%Vals AIAs of 10/25/2026
Code MigrationCode Quality · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 4.496 / 5Qwen 3.8 27B: 4.197 / 5Vals AIAs of 10/18/2026
EMBDCF · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 65.3%Qwen 3.8 27B: 50%Vals AIAs of 10/20/2026
Finance Agent (v2)Comparables · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 42.6%Qwen 3.8 27B: 38.9%Vals AIAs of 10/20/2026
IOIIOI 2026 · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 90.7%Qwen 3.8 27B: 36.8%Vals AIAs of 10/24/2026
Legal Research BenchBusiness & Commercial · All-pass · higher is betterTest setup not documented as identicalGPT-6.1 Sol: 39.5%Qwen 3.8 27B: 41.9%Vals AIAs of 10/26/2026