Published task benchmark

Commercial Loan Covenant Monitoring

Given loan terms and current financial statements, compute covenant ratios, compare them with thresholds, and flag current or near-term breaches.

Compare measured quality, workload cost, and latency across model tiers. Results are task-specific and directional—not a general model ranking.

Finance & AccountingAnalysisEvaluated 9/10/2026100 evaluation examplesFallback pricing checked September 11, 2026
Analysis

Commercial Loan Covenant Monitoring

Given loan terms and current financial statements, compute covenant ratios, compare them with thresholds, and flag current or near-term breaches.

Finance & AccountingPer loan
Not benchmarked
No result on this tier
Token profile pending
/100K
Run this tier to add it
Nemotron Nano 9B v2
9B · Off-the-shelf
680 in · 1,809 out · 12.06s p50
$45.69 /100K
AWS Bedrock temporary list price
View token rates
AWS Bedrock · $0.0600 input · $0.2300 output /1M
31%
DeepSeek V3
Hosted · Off-the-shelf
572 in · 1,105 out · 9.83s p50
$123 /100K
Best current price · TrustedRouter
Compare 2 router rates · observed 9/11/2026
TrustedRouter · $0.3376 input · $0.9390 output /1M
OpenRouter · $0.2900 input · $1.14 output /1M
53%
GPT-5.4
Hosted · Off-the-shelf
551 in · 1,488 out · 12.08s p50
$2370 /100K
Best current price · OpenRouter
Compare 2 router rates · observed 9/11/2026
OpenRouter · $2.50 input · $15.00 output /1M
TrustedRouter · $2.64 input · $15.82 output /1M
54%
All 12 model results and methodology

Generated 100/100 examples

ModelTierQualityJudgedScenario cost
Nemotron Nano 9B v2small31%100/100$45.69
Qwen3 235B A22Bmid43%100/100$313
DeepSeek V3mid53%100/100$123
Mistral Large 2407mid36%100/100$271
Arcee Trinity Large Thinkingmid25%100/100$162
GPT-5.4frontier54%100/100$2370
Claude Opus 4.7frontier54%100/100$5394
Gemini 3.1 Pro Previewfrontier3%100/100$2527
Gemma 4 E4B ITsmall25%100/100$17.95
Granite 4.1 8Bsmall17%100/100$15.64
Ministral 8B Instruct 2410small9%100/100$28.07
Qwen3 4B Instruct 2507small16%100/100$118

LLM-judge pass rate on 100 synthetic examples. Generator: gpt-5.2. Judge: gpt-5.2. Evaluated 2026-09-10T08:58:02.679Z.

Directional: measured on a synthetic eval set generated by drydock. Cost/latency are not yet captured for taskrouter-run benchmarks.

drydock

Have a neighboring workload?

Describe it in Task Explorer. If it is not measured yet, you can request a benchmark and follow future results.

Explore another task