← All comparisons

Llama 3.1 Nemotron Ultra 253B v1 (Reasoning) vs OLMo 2 32B

NVIDIA vs Allen Institute for AI — side-by-side benchmark comparison

Llama 3.1 Nemotron Ultra 253B v1 (Reasoning)OLMo 2 32B
Intelligence Index15.010.6
Coding Index13.12.7
Math Index63.73.3
Output speed (tok/s)52.30.0
Blended price ($/1M)$0.90$0.00
Time to first token (s)0.76s0.00s
aime74.7%—
aime 2563.7%3.3%
artificial analysis coding index13.102.70
artificial analysis intelligence index15.0010.60
artificial analysis math index63.703.30
gpqa72.8%32.8%
hle8.1%3.7%
ifbench38.2%38.1%
lcr7.3%0.0%
livecodebench64.1%6.8%
math 50095.2%—
mmlu pro82.5%51.1%
scicode34.7%8.0%
tau211.4%0.0%
terminalbench hard2.3%0.0%

Benchmark data from Artificial Analysis.