Analysis
Airline Itinerary Change Agent
Rebook or cancel flights while applying fare, baggage, loyalty, and disruption policies.
Customer service and account operationsPer itinerary
Not benchmarked
No result on this tier
Token profile pending
— /100K
Run this tier to add it
—
Gemma 4 E4B IT
E4B · Off-the-shelf
926 in · 986 out · 15.67s p50
$12.36 /100K
Current price · TrustedRouter
View token rates · observed 9/11/2026
TrustedRouter · $0.0211 input · $0.1055 output /1M
50%
Cheapest
Mistral Large 2407
123B · Off-the-shelf
1,019 in · 1,231 out · 36.28s p50
$249 /100K
Best current price · TrustedRouter
Compare 2 router rates · observed 9/11/2026
TrustedRouter · $0.5275 input · $1.58 output /1M
OpenRouter · $2.00 input · $6.00 output /1M
44%
GPT-5.4
Hosted · Off-the-shelf
817 in · 822 out · 8.54s p50
$1437 /100K
Best current price · OpenRouter
Compare 2 router rates · observed 9/11/2026
OpenRouter · $2.50 input · $15.00 output /1M
TrustedRouter · $2.64 input · $15.82 output /1M
88%
Best accuracy
All 12 model results and methodology
Generated 100/100 examples
| Model | Tier | Quality | Judged | Scenario cost |
|---|---|---|---|---|
| Nemotron Nano 9B v2 | small | 30% | 100/100 | $41.48 |
| Qwen3 235B A22B | mid | 40% | 100/100 | $264 |
| DeepSeek V3 | mid | 42% | 100/100 | $101 |
| Mistral Large 2407 | mid | 44% | 100/100 | $249 |
| Arcee Trinity Large Thinking | mid | 24% | 100/100 | $153 |
| GPT-5.4 | frontier | 88% | 100/100 | $1437 |
| Claude Opus 4.7 | frontier | 70% | 100/100 | $4239 |
| Gemini 3.1 Pro Preview | frontier | 51% | 100/100 | $1727 |
| Gemma 4 E4B IT | small | 50% | 100/100 | $12.36 |
| Granite 4.1 8B | small | 33% | 100/100 | $14.21 |
| Ministral 8B Instruct 2410 | small | 30% | 100/100 | $29.45 |
| Qwen3 4B Instruct 2507 | small | 21% | 100/100 | $112 |
LLM-judge pass rate on 100 synthetic examples. Generator: gpt-5.2. Judge: gpt-5.2. Evaluated 2026-09-10T20:53:03.109Z.
Directional: measured on a synthetic eval set generated by drydock. Cost/latency are not yet captured for taskrouter-run benchmarks.
drydock