Analysis

Inspect failed jobs/logs, distinguish flaky from real failures, patch, and rerun the right checks.

Software engineering, data, and DevOpsPer failure

Not benchmarked

No result on this tier

Token profile pending

— /100K

Run this tier to add it

—

Gemma 4 E4B IT

E4B · Off-the-shelf

737 in · 701 out · 11.12s p50

$8.95 /100K

Current price · TrustedRouter

View token rates · observed 9/11/2026

TrustedRouter · $0.0211 input · $0.1055 output /1M

59%

Cheapest

Qwen3 235B A22B

235B (22B active) · Off-the-shelf

666 in · 1,167 out · 9.61s p50

$243 /100K

Best current price · OpenRouter

Compare 2 router rates · observed 9/11/2026

OpenRouter · $0.4550 input · $1.82 output /1M

TrustedRouter · $0.6330 input · $2.11 output /1M

68%

GPT-5.4

Hosted · Off-the-shelf

636 in · 687 out · 8.03s p50

$1190 /100K

Best current price · OpenRouter

Compare 2 router rates · observed 9/11/2026

OpenRouter · $2.50 input · $15.00 output /1M

TrustedRouter · $2.64 input · $15.82 output /1M

98%

Best accuracy

All 12 model results and methodology

Generated 100/100 examples

Model	Tier	Quality	Judged	Scenario cost
Nemotron Nano 9B v2	small	36%	100/100	$37.93
Qwen3 235B A22B	mid	68%	100/100	$243
DeepSeek V3	mid	62%	100/100	$71.88
Mistral Large 2407	mid	66%	100/100	$182
Arcee Trinity Large Thinking	mid	38%	100/100	$139
GPT-5.4	frontier	98%	100/100	$1190
Claude Opus 4.7	frontier	90%	100/100	$4426
Gemini 3.1 Pro Preview	frontier	43%	99/100	$2250
Gemma 4 E4B IT	small	59%	100/100	$8.95
Granite 4.1 8B	small	42%	100/100	$10.91
Ministral 8B Instruct 2410	small	34%	100/100	$20.46
Qwen3 4B Instruct 2507	small	27%	100/100	$89.33

LLM-judge pass rate on 100 synthetic examples. Generator: gpt-5.2. Judge: gpt-5.2. Evaluated 2026-09-11T09:57:02.199Z.

Directional: measured on a synthetic eval set generated by drydock. Cost/latency are not yet captured for taskrouter-run benchmarks.

drydock

Have a neighboring workload?