Model profile
Gemini 2 Flash
Googledeveloper
2024-12-11release date
#130 / 267overall rank
7eval lineages
Evidence summary
Gemini 2 Flash has an estimated overall rank of #130; its 90% source-sensitivity interval is #78–#194. Its behavior-only rank is #141; company governance moves the combined estimate to #130. Published evidence spans 7 evals and 5 of 7 behavior components. Its strongest relative result is AILuminate General Purpose AI Chat (sex_related_crimes, #9 of 32); its weakest is AILuminate General Purpose AI Chat (child_sexual_exploitation, #25 of 32).
Compare this model
Only models sharing at least one published sub-eval are listed.
Official and reference links
- Official model documentation ↗Family-level model document · google · first party
- Release source ↗direct research preview identity date
Published eval results
Rank is within that sub-eval. Black marks the observed result; the grey dot marks the value implied by the global rank. Values stay on each source’s native scale.
| Eval / sub-eval | Rank | Value | Distribution | Better | Source |
|---|---|---|---|---|---|
| AILuminate General Purpose AI Chatchild_sexual_exploitation | #25 / 32 | 27.2 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatdefamation | #18 / 32 | 17.2 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chathate | #13 / 32 | 7.1 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatindiscriminate_weapons_cbrne | #14 / 32 | 20 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatintellectual_property | #24 / 32 | 19.7 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatnon_violent_crimes | #16 / 32 | 16 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatprivacy | #18 / 32 | 13.2 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatsex_related_crimes | #9 / 32 | 8.2 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatsexual_content | #10 / 32 | 16.3 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatspecialized_advice | #17 / 32 | 16.5 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatsuicide_self_harm | #16 / 32 | 12.8 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatviolent_crimes | #13 / 32 | 16.3 | ↓ lower | Source ↗official | |
| AIRBench 2024 Safety Scenariossafety_scenarios | #41 / 80 | 0.662 | ↑ higher | Source ↗official | |
| HELM Safetyanthropic_red_team | #23 / 80 | 0.994 | ↑ higher | Source ↗official | |
| HELM Safetybbq | #23 / 80 | 0.954 | ↑ higher | Source ↗official | |
| HELM Safetyharmbench | #51 / 80 | 0.662 | ↑ higher | Source ↗official | |
| HELM Safetysimple_safety_tests | #43 / 80 | 0.985 | ↑ higher | Source ↗official | |
| HELM Safetyxstest | #48 / 80 | 0.953 | ↑ higher | Source ↗official |