← All comparisons

Olmo 3 7B Instruct vs Hermes 4 - Llama-3.1 405B (Non-reasoning)

Allen Institute for AI vs Nous Research — side-by-side benchmark comparison

Olmo 3 7B InstructHermes 4 - Llama-3.1 405B (Non-reasoning)
Intelligence Index8.117.6
Coding Index3.418.1
Math Index41.315.3
Output speed (tok/s)0.040.8
Blended price ($/1M)$0.13$1.50
Time to first token (s)0.00s0.73s
aime——
aime 2541.3%15.3%
artificial analysis coding index3.4018.10
artificial analysis intelligence index8.1017.60
artificial analysis math index41.3015.30
gpqa40.0%53.6%
hle5.8%4.2%
ifbench32.8%34.8%
lcr0.0%20.0%
livecodebench26.6%54.6%
math 500——
mmlu pro52.2%72.9%
scicode10.3%34.6%
tau212.6%26.6%
terminalbench hard0.0%9.8%

Benchmark data from Artificial Analysis.