Evaluation profile
CAIS Risk Index
7sub-evals
6.35%total index weight
4components
Within-component eval weight: Responsible agency 15.4% · Misuse resistance 4.43% · Truthfulness 14.9% · Fairness 13.5%.
Model score (lower is better)Predicted score
About this eval
Frontier-model risk across prompt injection, biological-weapons assistance, political manipulation, overconfidence, deception, and harmful propensities.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction | Total index weight | Within-component weight |
|---|---|---|---|---|
| agent_red_teamingaisafetyindex/cais-risk-index.csv:agent_red_teamingMeasures whether an agent resists prompt injections and adversarial instructions while using tools. | responsible_agency_control:1.000cais-agent-red-teaming | Lower is better | 1.29% | Responsible agency 8.61% |
| bioweapons_assistanceaisafetyindex/cais-risk-index.csv:bioweapons_assistanceMeasures whether the model avoids providing assistance that could enable the development or use of biological weapons. | ordinary_harm_misuse_resistance:1.000cais-bioweapons-assistance | Lower is better | 0.443% | Misuse resistance 4.43% |
| hle_overconfidenceaisafetyindex/cais-risk-index.csv:hle_overconfidenceMeasures whether the model avoids expressing unjustified confidence when answering exceptionally difficult factual questions. | truthfulness_honesty:1.000cais-hle-overconfidence | Lower is better | 0.752% | Truthfulness 5.02% |
| machiavelliaisafetyindex/cais-risk-index.csv:machiavelliMeasures whether the model avoids deceptive and power-seeking actions while pursuing goals in interactive text games. | truthfulness_honesty:1.000machiavelli | Lower is better | 0.721% | Truthfulness 4.81% |
| maskaisafetyindex/cais-risk-index.csv:maskMeasures whether the model avoids lying when deception would help it satisfy an instructed goal. | truthfulness_honesty:1.000mask | Lower is better | 0.768% | Truthfulness 5.12% |
| political_manipulationaisafetyindex/cais-risk-index.csv:political_manipulationMeasures whether the model avoids generating targeted content intended to manipulate political beliefs or behavior. | fairness_nondiscrimination:1.000cais-political-manipulation | Lower is better | 1.35% | Fairness 13.5% |
| textquests_harmaisafetyindex/cais-risk-index.csv:textquests_harmMeasures whether the model avoids harmful actions while pursuing goals in interactive text-based environments. | responsible_agency_control:1.000cais-textquests-harm | Lower is better | 1.02% | Responsible agency 6.82% |
agent_red_teaming
Measures whether an agent resists prompt injections and adversarial instructions while using tools.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-fable-5 | 39.4 | official | |
| 2 | gpt-5.6-sol | 40.3 | official | |
| 3 | gpt-5.5 | 41.5 | official | |
| 4 | claude-opus-4.5 | 42.5 | official | |
| 5 | claude-opus-4.7 | 44.5 | official | |
| 6 | muse-spark-1.1 | 44.8 | official | |
| 7 | gemini-3.1-flash-lite | 45.4 | official | |
| 8 | claude-opus-4.8 | 47 | official | |
| 9 | gemini-3-pro-preview | 47.5 | official | |
| 10 | claude-opus-4.6 | 49.5 | official | |
| 11 | gpt-5.6-terra | 54.6 | official | |
| 12 | claude-sonnet-4.6 | 54.8 | official | |
| 13 | gpt-5.4 | 59.7 | official | |
| 14 | gpt-5.6-luna | 64.2 | official | |
| 15 | claude-sonnet-4.5 | 64.6 | official | |
| 16 | gemini-3.1-pro-preview | 68 | official | |
| 17 | gpt-5.2 | 68.8 | official | |
| 18 | claude-haiku-4.5 | 69.7 | official | |
| 19 | glm-5.2 | 71 | official | |
| 20 | gemini-3.5-flash | 72.3 | official | |
| 21 | gpt-5.4-mini | 74.3 | official | |
| 22 | glm-5.1 | 75.8 | official | |
| 23 | kimi-k3 | 76.6 | official | |
| 24 | gpt-5.4-nano | 81.7 | official | |
| 25 | gpt-5.1 | 82.1 | official | |
| 26 | grok-4.5 | 83.5 | official | |
| 27 | gpt-5-mini | 84.3 | official | |
| 28 | grok-4.20 | 84.6 | official | |
| 29 | grok-4 | 85.9 | official | |
| 30 | claude-sonnet-4 | 86.6 | official | |
| 31 | gpt-5-nano | 88.5 | official | |
| 32 | grok-4.3 | 90.1 | official | |
| 33 | grok-4.1-fast | 90.2 | official | |
| 34 | gpt-4o | 90.7 | official | |
| 34 | kimi-k2.6 | 90.7 | official | |
| 36 | deepseek-r1 | 90.9 | official | |
| 36 | kimi-k2.5 | 90.9 | official | |
| 38 | gemini-2.5-pro | 91.4 | official | |
| 39 | gemini-2.5-flash | 91.9 | official | |
| 40 | kimi-k2 | 92.8 | official | |
| 41 | gemini-3-flash-preview | 92.9 | official | |
| 42 | deepseek-v3.2 | 93 | official | |
| 43 | gemini-2.5-flash-lite | 96.8 | official |
bioweapons_assistance
Measures whether the model avoids providing assistance that could enable the development or use of biological weapons.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-fable-5 | 0 | official | |
| 2 | claude-sonnet-4.5 | 12 | official | |
| 3 | claude-opus-4.7 | 19.2 | official | |
| 4 | muse-spark-1.1 | 22.3 | official | |
| 5 | claude-opus-4.8 | 26.4 | official | |
| 6 | claude-sonnet-4.6 | 31.5 | official | |
| 7 | claude-opus-4.6 | 34.8 | official | |
| 8 | claude-opus-4.5 | 35.3 | official | |
| 9 | o3 | 47.7 | official | |
| 10 | grok-4.20 | 56.9 | official | |
| 11 | gpt-5.4 | 57.1 | official | |
| 12 | gpt-5.4-mini | 60 | official | |
| 13 | grok-4.3 | 60.1 | official | |
| 14 | gpt-5.5 | 60.5 | official | |
| 15 | gpt-5 | 62.6 | official | |
| 16 | gpt-5.2 | 62.9 | official | |
| 17 | glm-5.2 | 63.8 | official | |
| 17 | grok-4.1-fast | 63.8 | official | |
| 19 | grok-4-fast | 64.1 | official | |
| 20 | grok-4 | 64.2 | official | |
| 21 | gpt-5.6-sol | 64.3 | official | |
| 22 | gpt-5.6-terra | 66.2 | official | |
| 23 | gpt-5.1 | 66.3 | official | |
| 24 | o4-mini | 67.6 | official | |
| 25 | o1 | 68.1 | official | |
| 26 | gpt-5.6-luna | 68.8 | official | |
| 27 | gpt-5-nano | 71.3 | official | |
| 28 | gemini-3.1-pro-preview | 73.5 | official | |
| 29 | gemini-3-flash-preview | 75.3 | official | |
| 30 | gpt-5-mini | 78.2 | official | |
| 31 | gemini-3.5-flash | 78.3 | official | |
| 31 | gpt-5.4-nano | 78.3 | official | |
| 33 | grok-4.5 | 79.2 | official | |
| 34 | claude-haiku-4.5 | 84.9 | official | |
| 35 | kimi-k3 | 87.2 | official | |
| 36 | o3-mini | 88.3 | official | |
| 37 | kimi-k2.6 | 88.4 | official | |
| 38 | glm-5.1 | 89 | official | |
| 39 | gemini-3.1-flash-lite | 89.8 | official | |
| 40 | gpt-4o | 92.7 | official | |
| 41 | kimi-k2 | 93 | official | |
| 42 | kimi-k2.5 | 93.2 | official | |
| 43 | deepseek-v3.2 | 94 | official | |
| 44 | gemini-2.5-pro | 94.1 | official | |
| 45 | deepseek-r1 | 94.4 | official | |
| 46 | gemini-2.5-flash | 95.9 | official | |
| 47 | gemini-2.5-flash-lite | 96.3 | official | |
| 48 | gemini-3-pro-preview | 100 | official |
hle_overconfidence
Measures whether the model avoids expressing unjustified confidence when answering exceptionally difficult factual questions.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | muse-spark-1.1 | 25.3 | official | |
| 2 | claude-fable-5 | 25.5 | official | |
| 3 | claude-opus-4.8 | 26.4 | official | |
| 4 | claude-opus-4.7 | 28.1 | official | |
| 5 | grok-4.5 | 30.8 | official | |
| 6 | gpt-5.4 | 40.9 | official | |
| 7 | gpt-5.5 | 41.8 | official | |
| 8 | glm-5.2 | 42 | official | |
| 9 | grok-4.3 | 42.5 | official | |
| 10 | claude-sonnet-4.6 | 43.1 | official | |
| 11 | gpt-5.2 | 43.7 | official | |
| 12 | kimi-k3 | 44.7 | official | |
| 13 | grok-4.20 | 45.8 | official | |
| 14 | claude-opus-4.6 | 46 | official | |
| 15 | gpt-5.6-sol | 46.7 | official | |
| 16 | gpt-5.4-nano | 47.1 | official | |
| 17 | gpt-5.1 | 47.8 | official | |
| 18 | gpt-5 | 50 | official | |
| 19 | gemini-3.1-pro-preview | 50.3 | official | |
| 20 | gpt-5.6-terra | 50.7 | official | |
| 21 | gemini-3.5-flash | 53.5 | official | |
| 22 | gemini-3-flash-preview | 55.1 | official | |
| 23 | gpt-5.4-mini | 55.2 | official | |
| 24 | gpt-5.6-luna | 55.7 | official | |
| 25 | grok-4 | 56.4 | official | |
| 26 | claude-opus-4.5 | 56.9 | official | |
| 27 | gemini-3-pro-preview | 57.2 | official | |
| 28 | kimi-k2.6 | 58.2 | official | |
| 29 | glm-5.1 | 58.5 | official | |
| 30 | deepseek-v3.2 | 58.8 | official | |
| 31 | claude-haiku-4.5 | 60.5 | official | |
| 32 | claude-sonnet-4.5 | 64.1 | official | |
| 33 | gpt-5-mini | 65 | official | |
| 34 | kimi-k2 | 67.8 | official | |
| 35 | grok-4-fast | 68.7 | official | |
| 36 | gemini-2.5-pro-exp | 71 | official | |
| 37 | gemini-2.5-pro | 72 | official | |
| 38 | grok-4.1-fast | 72.1 | official | |
| 39 | gemini-3.1-flash-lite | 72.8 | official | |
| 40 | deepseek-r1 | 73 | official | |
| 41 | claude-3.7-sonnet | 75 | official | |
| 41 | claude-sonnet-4 | 75 | official | |
| 43 | kimi-k2.5 | 75.9 | official | |
| 44 | gemini-2.5-flash | 80 | official | |
| 44 | gpt-5-nano | 80 | official | |
| 44 | o3-mini | 80 | official | |
| 47 | o1 | 83 | official | |
| 48 | gemini-2.5-flash-lite | 83.4 | official | |
| 49 | gpt-4o | 89 | official |
machiavelli
Measures whether the model avoids deceptive and power-seeking actions while pursuing goals in interactive text games.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | gpt-5-nano | 79.4 | official | |
| 2 | claude-fable-5 | 80.2 | official | |
| 3 | gpt-5.6-terra | 80.4 | official | |
| 4 | claude-opus-4.5 | 80.8 | official | |
| 5 | grok-4.1-fast | 81.3 | official | |
| 6 | claude-haiku-4.5 | 81.5 | official | |
| 7 | claude-sonnet-4.5 | 81.6 | official | |
| 8 | gpt-5.5 | 81.7 | official | |
| 9 | gpt-5.6-sol | 82.7 | official | |
| 10 | claude-opus-4.8 | 83.7 | official | |
| 11 | gpt-5 | 83.8 | official | |
| 11 | grok-4.20 | 83.8 | official | |
| 13 | gpt-5-mini | 84.2 | official | |
| 13 | kimi-k2 | 84.2 | official | |
| 15 | claude-opus-4.6 | 84.4 | official | |
| 16 | grok-4.3 | 84.7 | official | |
| 17 | claude-sonnet-4.6 | 84.9 | official | |
| 18 | deepseek-r1 | 85.1 | official | |
| 19 | claude-opus-4.7 | 85.7 | official | |
| 20 | gpt-5.2 | 86.5 | official | |
| 21 | glm-5.1 | 86.7 | official | |
| 22 | gpt-5.6-luna | 87.9 | official | |
| 23 | muse-spark-1.1 | 88.2 | official | |
| 24 | gpt-5.4-nano | 88.5 | official | |
| 25 | gemini-3.1-pro-preview | 89.2 | official | |
| 26 | gemini-2.5-flash | 89.4 | official | |
| 27 | deepseek-v3.2 | 89.6 | official | |
| 27 | gpt-5.1 | 89.6 | official | |
| 29 | grok-4.5 | 90.9 | official | |
| 30 | kimi-k2.6 | 92.1 | official | |
| 31 | gemini-3.5-flash | 92.3 | official | |
| 31 | glm-5.2 | 92.3 | official | |
| 33 | kimi-k3 | 92.6 | official | |
| 34 | gpt-5.4 | 92.9 | official | |
| 35 | grok-4 | 93.4 | official | |
| 36 | kimi-k2.5 | 93.5 | official | |
| 37 | gemini-2.5-pro | 95.3 | official | |
| 37 | gemini-3-flash-preview | 95.3 | official | |
| 39 | gpt-5.4-mini | 95.6 | official | |
| 40 | gemini-3.1-flash-lite | 95.8 | official | |
| 41 | grok-4-fast | 96.2 | official | |
| 42 | gpt-4o | 99.6 | official | |
| 43 | gemini-3-pro-preview | 99.8 | official | |
| 44 | o3-mini | 105 | official | |
| 45 | gemini-2.5-flash-lite | 109.4 | official |
mask
Measures whether the model avoids lying when deception would help it satisfy an instructed goal.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | muse-spark-1.1 | 3.8 | official | |
| 2 | claude-sonnet-4.5 | 3.9 | official | |
| 3 | claude-haiku-4.5 | 4.6 | official | |
| 3 | gpt-5.6-sol | 4.6 | official | |
| 5 | claude-opus-4.5 | 5.9 | official | |
| 6 | gpt-5.6-terra | 6.7 | official | |
| 7 | claude-fable-5 | 7.2 | official | |
| 8 | gpt-5.6-luna | 7.3 | official | |
| 9 | claude-opus-4.8 | 8.1 | official | |
| 10 | kimi-k3 | 9.4 | official | |
| 11 | gpt-5.4 | 9.7 | official | |
| 12 | gpt-5.5 | 9.9 | official | |
| 13 | claude-sonnet-4 | 10.7 | official | |
| 14 | claude-sonnet-4.6 | 11.7 | official | |
| 15 | gpt-5.1 | 12 | official | |
| 16 | gpt-5-nano | 12.3 | official | |
| 17 | claude-opus-4.6 | 12.7 | official | |
| 18 | gpt-5.2 | 12.9 | official | |
| 19 | o3 | 15.5 | official | |
| 20 | glm-5.1 | 15.8 | official | |
| 21 | gpt-5.4-mini | 15.9 | official | |
| 22 | claude-3.7-sonnet | 16.9 | official | |
| 23 | claude-opus-4.7 | 17.1 | official | |
| 24 | kimi-k2.5 | 18.5 | official | |
| 25 | gpt-5 | 20.7 | official | |
| 26 | o4-mini | 21.4 | official | |
| 27 | gpt-5-mini | 21.7 | official | |
| 28 | kimi-k2.6 | 23.2 | official | |
| 29 | gpt-5.4-nano | 23.7 | official | |
| 30 | glm-5.2 | 25.8 | official | |
| 31 | grok-4.5 | 27.9 | official | |
| 32 | kimi-k2 | 30.7 | official | |
| 33 | o1 | 40.7 | official | |
| 34 | grok-4.20 | 41 | official | |
| 35 | grok-4 | 41.7 | official | |
| 36 | deepseek-r1 | 42.7 | official | |
| 37 | gemini-2.5-pro | 44.3 | official | |
| 38 | gemini-2.5-pro-exp | 46.9 | official | |
| 39 | grok-4.3 | 48.3 | official | |
| 40 | gemini-2.5-flash | 50.9 | official | |
| 41 | o3-mini | 51.1 | official | |
| 42 | deepseek-v3.2 | 53 | official | |
| 43 | gemini-3.1-pro-preview | 53.8 | official | |
| 44 | gemini-2.5-flash-lite | 54 | official | |
| 45 | gemini-3.1-flash-lite | 55.8 | official | |
| 46 | gemini-3.5-flash | 57.4 | official | |
| 47 | gemini-3-pro-preview | 58 | official | |
| 48 | gemini-3-flash-preview | 61.7 | official | |
| 49 | grok-4-fast | 70 | official | |
| 50 | grok-4.1-fast | 71 | official | |
| 51 | gpt-4o | 78 | official |
political_manipulation
Measures whether the model avoids generating targeted content intended to manipulate political beliefs or behavior.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | muse-spark-1.1 | 27.9 | official | |
| 2 | grok-4-fast | 30.8 | official | |
| 3 | grok-4.1-fast | 32.5 | official | |
| 4 | gpt-5 | 35.3 | official | |
| 5 | gemini-2.5-pro | 37.3 | official | |
| 6 | gpt-5.5 | 39 | official | |
| 6 | o3 | 39 | official | |
| 8 | claude-fable-5 | 39.8 | official | |
| 9 | grok-4.20 | 39.9 | official | |
| 10 | gpt-5.6-sol | 41.7 | official | |
| 11 | gemini-3.1-pro-preview | 43.3 | official | |
| 12 | kimi-k2.6 | 43.6 | official | |
| 13 | gpt-5.6-luna | 44.7 | official | |
| 14 | gpt-5.6-terra | 46.2 | official | |
| 15 | gpt-5.1 | 47.9 | official | |
| 16 | claude-opus-4.7 | 48.2 | official | |
| 17 | claude-opus-4.5 | 49.4 | official | |
| 18 | grok-4 | 49.5 | official | |
| 19 | glm-5.2 | 50 | official | |
| 20 | grok-4.5 | 50.2 | official | |
| 21 | grok-4.3 | 51.6 | official | |
| 22 | claude-opus-4.6 | 51.7 | official | |
| 23 | gemini-3.1-flash-lite | 51.9 | official | |
| 24 | kimi-k3 | 52.4 | official | |
| 25 | gemini-3.5-flash | 53.4 | official | |
| 26 | claude-opus-4.8 | 53.8 | official | |
| 27 | gpt-5.2 | 57 | official | |
| 28 | gpt-5.4 | 59.4 | official | |
| 29 | gpt-4o | 59.6 | official | |
| 30 | gpt-5.4-mini | 61.1 | official | |
| 31 | claude-haiku-4.5 | 62.2 | official | |
| 32 | gpt-5.4-nano | 63.3 | official |
textquests_harm
Measures whether the model avoids harmful actions while pursuing goals in interactive text-based environments.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | gpt-5-nano | 2 | official | |
| 2 | grok-4.3 | 3.9 | official | |
| 3 | grok-4.1-fast | 9.1 | official | |
| 4 | grok-4.20 | 9.9 | official | |
| 5 | gpt-4o | 11.2 | official | |
| 5 | grok-4.5 | 11.2 | official | |
| 7 | gemini-2.5-flash | 11.7 | official | |
| 8 | o3-mini | 13.4 | official | |
| 9 | kimi-k2 | 14.6 | official | |
| 10 | gpt-5.4-nano | 15.2 | official | |
| 10 | muse-spark-1.1 | 15.2 | official | |
| 12 | deepseek-r1 | 15.4 | official | |
| 12 | gpt-5.4-mini | 15.4 | official | |
| 14 | gemini-3.1-flash-lite | 15.6 | official | |
| 15 | gemini-2.5-pro | 15.9 | official | |
| 16 | claude-sonnet-4 | 16 | official | |
| 17 | claude-sonnet-4.6 | 16.5 | official | |
| 18 | claude-haiku-4.5 | 16.6 | official | |
| 19 | gpt-5.1 | 16.7 | official | |
| 20 | kimi-k3 | 17 | official | |
| 21 | claude-sonnet-4.5 | 17.1 | official | |
| 22 | gpt-5 | 17.2 | official | |
| 23 | gpt-5-mini | 17.5 | official | |
| 24 | gemini-3-flash-preview | 17.8 | official | |
| 25 | glm-5.2 | 18.1 | official | |
| 26 | o4-mini | 18.6 | official | |
| 27 | o3 | 18.7 | official | |
| 28 | gemini-3.1-pro-preview | 18.8 | official | |
| 29 | grok-4-fast | 19.8 | official | |
| 30 | claude-opus-4.5 | 20.3 | official | |
| 31 | kimi-k2.6 | 20.4 | official | |
| 32 | gpt-5.2 | 20.5 | official | |
| 33 | gemini-3.5-flash | 20.7 | official | |
| 34 | kimi-k2.5 | 20.8 | official | |
| 35 | claude-opus-4.8 | 21 | official | |
| 36 | gemini-3-pro-preview | 21.4 | official | |
| 36 | gpt-5.5 | 21.4 | official | |
| 38 | deepseek-v3.2 | 21.6 | official | |
| 39 | claude-opus-4.7 | 21.8 | official | |
| 40 | glm-5.1 | 22 | official | |
| 41 | gpt-5.6-terra | 22.2 | official | |
| 42 | gemini-2.5-flash-lite | 22.8 | official | |
| 43 | claude-opus-4.6 | 23.2 | official | |
| 44 | claude-fable-5 | 23.7 | official | |
| 45 | gpt-5.6-luna | 23.8 | official | |
| 46 | grok-4 | 24.3 | official | |
| 47 | gpt-5.4 | 24.5 | official | |
| 48 | gpt-5.6-sol | 31.4 | official |