Gemini 3.1 Pro Previewvs.Qwen 3.8 27B

Which one is better?

Gemini 3.1 Pro Preview is clearly ahead in the overall index, but costs 50 times (input) and 4.8 times (output) as much per token.

IntelligenceOverall index, 0 to 100 · higher is better

Gemini 3.1 Pro Preview

Rank 36 to 44

Qwen 3.8 27B

Rank 44 to 57

SpeedTokens per second · higher is better

145

Gemini 3.1 Pro Preview

fairly fast

Qwen 3.8 27B

slow

PriceDollars per 1M tokens · lower is better

$4.50

Gemini 3.1 Pro Preview

expensive

$0.66

Qwen 3.8 27B

fairly cheap

Gemini 3.1 Pro Preview

Subscription: No Google subscription includes this model. Through other providers from ≈ €6.36 a month (ElevenLabs Starter, with limited credits). 26 plans in Germany.
Via API: $2 input, $12 output per 1M tokens, unchanged since our price tracking began on Aug 14, 2026.

Qwen 3.8 27B

Subscription: No Alibaba subscription includes this model. Through other providers from ≈ €1.06 a month (Command Code Go, with limited credits). 10 plans in Germany.
Via API: $0.04 input, $2.50 output per 1M tokens according to the provider; the price index has listed it at $0.42 and $3 since Oct 27, 2026.

Overall index with data through Oct 29, 2026 · Prices from Oct 29, 2026 · Speed: measured by Artificial Analysis · Subscriptions as of Oct 24, 2026 · Subscription prices incl. VAT

Where is the difference?

Values from 0 to 100 per task area. We only name a winner when the gap is large enough and there are enough tests.

Gemini 3.1 Pro Preview ahead: 4Qwen 3.8 27B ahead: 1level: 4too little data: 1of 10 areas

AreaGemini 3.1 Pro PreviewQwen 3.8 27B0 to 100, higher is betterResult

Areas in the overall index

Codinglevel1113

ReasoningGemini 3.1 Pro Preview ahead506

MathematicsGemini 3.1 Pro Preview ahead168

KnowledgeGemini 3.1 Pro Preview ahead475

Agentslevel2421

Generalonly Gemini 3.1 Pro Preview has a score29not enough data

MultimodalGemini 3.1 Pro Preview ahead4213

Specialist fields

FinanceQwen 3.8 27B ahead1034

Legallevel1623

Healthlevel333

Show 4 more areas that do not count toward the overall index

AreaGemini 3.1 Pro PreviewQwen 3.8 27B0 to 100, higher is betterResult

Cybersecurityonly Qwen 3.8 27B has a scorenot enough data19

Searchonly Gemini 3.1 Pro Preview has a score32not enough data

Productivityonly Gemini 3.1 Pro Preview has a score30not enough data

Educationonly Gemini 3.1 Pro Preview has a score12not enough data

See for yourself: same task, one attempt

The same prompt for both models. We show the latest attempt without fixes; it does not count toward the overall index.

Pelican on a bicycle

A community classic for free-form SVG drawing.

Gemini 3.1 Pro Preview

The response reached the token limit. No complete recording is available.

Reasoning (requested): high

Current attempt

Run date: Oct 6, 2026
API provider: openrouter
Requested API model: google/gemini-3.1-pro-preview
API model reported in response: google/gemini-3.1-pro-preview
Reasoning setting: high
Pinned endpoint: Google AI Studio | google/gemini-3.1-pro-preview-20260219
Temperature: Provider default
Token limit including reasoning: 8,192 tokens
Test protocol: community-visual-tests-v1
Technical checks: Failed
Response time: 63.8 s

Not yet visually reviewed. A successful recording does not confirm correct physics or full compliance with the prompt.

Earlier attempts (3)

Oct 2, 2026API request failedReasoning setting: high
Oct 1, 2026API request failedReasoning setting: high
Sep 7, 2026API request failedReasoning setting: high

Qwen 3.8 27B

Not tested yet

Prompt and test conditions

Generate an SVG of a pelican riding a bicycle

Token limit including reasoning: 8,192 tokens

Task origin (Simon Willison)

SVG short test

One SVG task with the same fixed settings for every model, without hidden thinking.

Shared test round: 43b8c5c46830

Gemini 3.1 Pro Preview

Unavailable in this round

Qwen 3.8 27B

All data side by side

The better value is bold. For the overall index only when the gap is clear.

Metric	Gemini 3.1 Pro Preview	Qwen 3.8 27B
Intelligence
Overall index0 to 100, higher is better	26	11
Rank	Rank 36 to 44 of 95	Rank 44 to 57 of 95
Price per 1M tokens
Inputwhat you send	$2above 200,000 tokens: $4	$0.04
Outputwhat the model writes	$12above 200,000 tokens: $18	$2.50
Speed
Tokens per second	145	40
Technical
Context windowhow much text the model reads at once	1.05 M tokens≈ 1,570 A4 pages	262,144 tokens≈ 390 A4 pages
Knowledge cutoff	n/a	n/a
Released	Feb 19, 2026	Aug 14, 2026
Availability	Preview	Active
Open weights	no	yes

Prices without caching or batch discounts · Page count as a rule of thumb from Artificial Analysis

Shared benchmarks

Single results from the same benchmark version, task and metric. A value is bold only with the same test setup and a clear gap.

A: Gemini 3.1 Pro PreviewB: Qwen 3.8 27B

Code MigrationOverall · higher is betterTest setup not documented as identical17.3%14.2%Vals AIAs of 10/18/2026

EMBOverall · higher is betterTest setup not documented as identical52.6%59.7%Vals AIAs of 10/20/2026

Finance Agent (v2)Overall · higher is betterTest setup not documented as identical43%48.6%Vals AIAs of 10/20/2026

GPQA DiamondOverall · higher is betterTest setup not documented as identical95.5%88.9%Vals AIAs of 09/01/2026

IOIOverall · higher is betterTest setup not documented as identical51.8%39.1%Vals AIAs of 10/24/2026

Legal Research BenchOverall · All-pass · higher is betterTest setup not documented as identical20.7%36.1%Vals AIAs of 10/26/2026

LegalBenchOverall · higher is betterTest setup not documented as identical87.4%82.4%Vals AIAs of 10/13/2026

LiveCodeBenchOverall · higher is betterTest setup not documented as identical88.5%84%Vals AIAs of 09/01/2026

MedCodeOverall · higher is betterTest setup not documented as identical59.1%28.7%Vals AIAs of 10/13/2026

MedScribeOverall · higher is betterTest setup not documented as identical76.1%83.8%Vals AIAs of 10/13/2026

Show 54 more benchmarks

MMLU ProOverall · higher is betterTest setup not documented as identical91%84.3%Vals AIAs of 09/01/2026

MMMU ProOverall · higher is betterTest setup not documented as identical88.2%83.9%Vals AIAs of 09/01/2026

MortgageTaxOverall · higher is betterTest setup not documented as identical69.4%64.9%Vals AIAs of 09/01/2026

ProgramBenchOverall · higher is betterTest setup not documented as identical0%0%Vals AIAs of 10/20/2026

ProofBench v1.1Overall · higher is betterTest setup not documented as identical26%16%Vals AIAs of 10/24/2026

SWE-bench VerifiedOverall · higher is betterTest setup not documented as identical78.8%86%Vals AIAs of 09/01/2026

Tax Agent BenchOverall · higher is betterTest setup not documented as identical9.3%30.9%Vals AIAs of 10/26/2026

TaxEval v2Overall · higher is betterTest setup not documented as identical72.9%70.9%Vals AIAs of 09/01/2026

Terminal-Bench 2.1Overall · higher is betterTest setup not documented as identical70.8%58.4%Vals AIAs of 10/01/2026

Terminal-Bench-ScienceOverall · higher is betterTest setup not documented as identical1.4%0%Vals AIAs of 10/25/2026

Vibe Code Bench v1.1Overall · higher is betterTest setup not documented as identical32%64.9%Vals AIAs of 10/26/2026

Code MigrationCLI · higher is betterTest setup not documented as identical10.8%2.2%Vals AIAs of 10/18/2026

EMBComps · higher is betterTest setup not documented as identical44.5%53.3%Vals AIAs of 10/20/2026

Finance Agent (v2)Adjustments · higher is betterTest setup not documented as identical38.4%47.3%Vals AIAs of 10/20/2026

GPQA DiamondFew-Shot CoT · higher is betterTest setup not documented as identical96%88.9%Vals AIAs of 09/01/2026

IOIIOI 2024 · higher is betterTest setup not documented as identical55.5%31.8%Vals AIAs of 10/24/2026

Legal Research BenchAdministrative / Regulatory · All-pass · higher is betterTest setup not documented as identical24.2%45.5%Vals AIAs of 10/26/2026

LegalBenchConclusion Tasks · higher is betterTest setup not documented as identical89.3%88.4%Vals AIAs of 10/13/2026

LiveCodeBenchEasy · higher is betterTest setup not documented as identical99.1%99.1%Vals AIAs of 09/01/2026

MMLU ProBiology · higher is betterTest setup not documented as identical95.3%91.5%Vals AIAs of 09/01/2026

MortgageTaxNumerical Extraction · higher is betterTest setup not documented as identical76.4%69.4%Vals AIAs of 09/01/2026

ProgramBenchAlmost Resolved · higher is betterTest setup not documented as identical1%1%Vals AIAs of 10/20/2026

SWE-bench Verified<15 min fix · higher is betterTest setup not documented as identical88.7%90.2%Vals AIAs of 09/01/2026

Tax Agent BenchControversy & Precedence Analysis · higher is betterTest setup not documented as identical0%6.7%Vals AIAs of 10/26/2026

TaxEval v2Correctness · higher is betterTest setup not documented as identical60.7%60.1%Vals AIAs of 09/01/2026

Terminal-Bench 2.1Easy · higher is betterTest setup not documented as identical91.7%50%Vals AIAs of 10/01/2026

Terminal-Bench-ScienceEarth Sciences · higher is betterTest setup not documented as identical0%0%Vals AIAs of 10/25/2026

Code MigrationCOBOL · higher is betterTest setup not documented as identical36.8%50%Vals AIAs of 10/18/2026

EMBDataroom · higher is betterTest setup not documented as identical79.2%79.7%Vals AIAs of 10/20/2026

Finance Agent (v2)All-Pass · higher is betterTest setup not documented as identical30.9%36.8%Vals AIAs of 10/20/2026

GPQA DiamondZero-Shot CoT · higher is betterTest setup not documented as identical94.9%88.9%Vals AIAs of 09/01/2026

IOIIOI 2025 · higher is betterTest setup not documented as identical46.8%48.5%Vals AIAs of 10/24/2026

Legal Research BenchAdministrative / Regulatory · Weighted partial credit · higher is betterTest setup not documented as identical79.2%88.5%Vals AIAs of 10/26/2026

LegalBenchInterpretation Tasks · higher is betterTest setup not documented as identical83.8%82%Vals AIAs of 10/13/2026

LiveCodeBenchHard · higher is betterTest setup not documented as identical76.6%65.7%Vals AIAs of 09/01/2026

MMLU ProBusiness · higher is betterTest setup not documented as identical93.4%90.2%Vals AIAs of 09/01/2026

MortgageTaxSemantic Extraction · higher is betterTest setup not documented as identical62.4%60.5%Vals AIAs of 09/01/2026

ProgramBenchFully Resolved · higher is betterTest setup not documented as identical0%0%Vals AIAs of 10/20/2026

SWE-bench Verified>4 hours · higher is betterTest setup not documented as identical33.3%66.7%Vals AIAs of 09/01/2026

Tax Agent BenchControversy & Precedence Analysis · higher is betterTest setup not documented as identical34.4%48.3%Vals AIAs of 10/26/2026

TaxEval v2Stepwise Reasoning · higher is betterTest setup not documented as identical85%81.6%Vals AIAs of 09/01/2026

Terminal-Bench 2.1Hard · higher is betterTest setup not documented as identical60%43.3%Vals AIAs of 10/01/2026

Terminal-Bench-ScienceEngineering Sciences · higher is betterTest setup not documented as identical0%0%Vals AIAs of 10/25/2026

Code MigrationCode Quality · higher is betterTest setup not documented as identical2.538 / 54.197 / 5Vals AIAs of 10/18/2026

EMBDCF · higher is betterTest setup not documented as identical46.4%50%Vals AIAs of 10/20/2026

Finance Agent (v2)Comparables · higher is betterTest setup not documented as identical36.7%38.9%Vals AIAs of 10/20/2026

IOIIOI 2026 · higher is betterTest setup not documented as identical53.2%36.8%Vals AIAs of 10/24/2026

Legal Research BenchBusiness & Commercial · All-pass · higher is betterTest setup not documented as identical23.3%41.9%Vals AIAs of 10/26/2026

LegalBenchIssue Tasks · higher is betterTest setup not documented as identical91.5%90.2%Vals AIAs of 10/13/2026

LiveCodeBenchMedium · higher is betterTest setup not documented as identical89.8%87.2%Vals AIAs of 09/01/2026

MMLU ProChemistry · higher is betterTest setup not documented as identical93.3%91.8%Vals AIAs of 09/01/2026

ProgramBenchRaw Pass Rate · higher is betterTest setup not documented as identical39.5%11.2%Vals AIAs of 10/20/2026

SWE-bench Verified1-4 hours · higher is betterTest setup not documented as identical42.9%69%Vals AIAs of 09/01/2026

Tax Agent BenchCurrent & Temporal Analysis · higher is betterTest setup not documented as identical9.8%43.1%Vals AIAs of 10/26/2026

Compare another matchup

First modelSecond model

More matchups

Compare Gemini 3.1 Pro Preview and Qwen 3.8 27B with other leading models.

View the complete LLM comparison

Gemini 3.1 Pro Previewvs.Qwen 3.8 27B

Gemini 3.1 Pro Preview

Qwen 3.8 27B

Where is the difference?

See for yourself: same task, one attempt

Pelican on a bicycle

SVG short test

Gemini 3.1 Pro Preview

Qwen 3.8 27B

All data side by side

Shared benchmarks

Compare another matchup

Related LLM comparisons