← All comparisons

Gemini 2.0 Flash (Feb '25) vs Llama 3.1 Tulu3 405B

Google vs Allen Institute for AI — side-by-side benchmark comparison

Gemini 2.0 Flash (Feb '25)Llama 3.1 Tulu3 405B
Intelligence Index18.514.1
Coding Index13.6—
Math Index21.7—
Output speed (tok/s)0.00.0
Blended price ($/1M)$0.26$0.00
Time to first token (s)0.00s0.00s
aime33.0%13.3%
aime 2521.7%—
artificial analysis coding index13.60—
artificial analysis intelligence index18.5014.10
artificial analysis math index21.70—
gpqa62.3%51.6%
hle5.3%3.5%
ifbench40.2%—
lcr28.3%—
livecodebench33.4%29.1%
math 50093.0%77.8%
mmlu pro77.9%71.6%
scicode33.3%30.2%
tau229.5%—
terminalbench hard3.8%—

Benchmark data from Artificial Analysis.