Evaluation profile
Emergent Collusion
1sub-evals
0.71%total index weight
1components
Within-component eval weight: Responsible agency 4.73%.
Model score (lower is better)Predicted score
About this eval
Spontaneous anti-competitive coordination in repeated multi-agent auctions.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction | Total index weight | Within-component weight |
|---|---|---|---|---|
| high_illegality_game_rateemergent-collusion/emergent-collusion.csv:high_illegality_game_rateMeasures how often autonomous buyer and seller agents spontaneously coordinate prices or market allocation in repeated auctions. | responsible_agency_control:1.000emergent-collusion | Lower is better | 0.71% | Responsible agency 4.73% |
high_illegality_game_rate
Measures how often autonomous buyer and seller agents spontaneously coordinate prices or market allocation in repeated auctions.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | grok-3 | 0.19 | official | |
| 2 | chatgpt-4o | 0.23 | official | |
| 3 | deepseek-v3 | 0.26 | official | |
| 4 | grok-3-mini | 0.3 | official | |
| 5 | claude-sonnet-4 | 0.32 | official | |
| 6 | claude-opus-4 | 0.36 | official | |
| 7 | gemini-2.5-pro | 0.39 | official | |
| 8 | mistral-medium-3 | 0.47 | official | |
| 9 | o3 | 0.51 | official | |
| 10 | qwen3-235b-a22b | 0.57 | official | |
| 11 | o4-mini | 0.62 | official | |
| 12 | deepseek-r1 | 0.71 | official | |
| 13 | grok-4 | 0.75 | official |