Evaluation profile
MACHIAVELLI
1sub-evals
0.76%total index weight
1components
Within-component eval weight: Truthfulness 5.07%.
Model score (lower is better)Predicted score
About this eval
Agentic behavior involving power-seeking, disutility, and ethical violations.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction | Total index weight | Within-component weight |
|---|---|---|---|---|
| deception_relative_random_pctmachiavelli/machiavelli-deception-union.csv:deception_relative_random_pctMeasures deceptive actions taken by the model while pursuing goals in interactive text games. | truthfulness_honesty:1.000machiavelli | Lower is better | 0.76% | Truthfulness 5.07% |
deception_relative_random_pct
Measures deceptive actions taken by the model while pursuing goals in interactive text games.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | gpt-5-nano | 79.4 | official | |
| 2 | claude-fable-5 | 80.2 | official | |
| 3 | gpt-5.6-terra | 80.4 | official | |
| 4 | claude-opus-4.5 | 80.8 | official | |
| 5 | grok-4.1-fast | 81.3 | official | |
| 6 | claude-haiku-4.5 | 81.5 | official | |
| 7 | claude-sonnet-4.5 | 81.6 | official | |
| 8 | gpt-5.5 | 81.7 | official | |
| 9 | gpt-5.6-sol | 82.7 | official | |
| 10 | claude-opus-4.8 | 83.7 | official | |
| 11 | gpt-5 | 83.8 | official | |
| 11 | grok-4.20 | 83.8 | official | |
| 13 | gpt-5-mini | 84.2 | official | |
| 13 | kimi-k2 | 84.2 | official | |
| 15 | claude-opus-4.6 | 84.4 | official | |
| 16 | grok-4.3 | 84.7 | official | |
| 17 | claude-sonnet-4.6 | 84.9 | official | |
| 18 | deepseek-r1 | 85.1 | official | |
| 19 | claude-opus-4.7 | 85.7 | official | |
| 20 | gpt-5.2 | 86.5 | official | |
| 21 | glm-5.1 | 86.7 | official | |
| 22 | gpt-5.6-luna | 87.9 | official | |
| 23 | muse-spark-1.1 | 88.2 | official | |
| 24 | gpt-5.4-nano | 88.5 | official | |
| 25 | gemini-3.1-pro-preview | 89.2 | official | |
| 26 | gemini-2.5-flash | 89.4 | official | |
| 27 | deepseek-v3.2 | 89.6 | official | |
| 27 | gpt-5.1 | 89.6 | official | |
| 29 | gpt-4 | 90 | official | |
| 30 | grok-4.5 | 90.9 | official | |
| 31 | kimi-k2.6 | 92.1 | official | |
| 32 | gemini-3.5-flash | 92.3 | official | |
| 32 | glm-5.2 | 92.3 | official | |
| 34 | gpt-5.4 | 92.9 | official | |
| 35 | llama-13b | 93 | official | |
| 36 | grok-4 | 93.4 | official | |
| 37 | kimi-k2.5 | 93.5 | official | |
| 38 | gpt-3.5-turbo | 94 | official | |
| 39 | gemini-2.5-pro | 95.3 | official | |
| 39 | gemini-3-flash-preview | 95.3 | official | |
| 41 | gpt-5.4-mini | 95.6 | official | |
| 42 | gemini-3.1-flash-lite | 95.8 | official | |
| 43 | grok-4-fast | 96.2 | official | |
| 44 | gpt-4o | 99.6 | official | |
| 45 | gemini-3-pro-preview | 99.8 | official | |
| 46 | llama-30b | 101 | official | |
| 47 | davinci | 105 | official | |
| 47 | llama-7b | 105 | official | |
| 47 | o3-mini | 105 | official | |
| 50 | gemini-2.5-flash-lite | 109.4 | official |