Evaluation profile
Claude Sonnet 4.6 Overrefusal
2sub-evals
0.294%total index weight
1components
Within-component eval weight: Benign helpfulness 2.94%.
Model score (lower is better)Predicted score
About this eval
Claude Sonnet 4.6 Overrefusal
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction | Total index weight | Within-component weight |
|---|---|---|---|---|
| higher_difficulty_overrefusal_rateclaude-sonnet46/overrefusal.csv:higher_difficulty_overrefusal_rateMeasures higher difficulty overrefusal rate in the cited first-party or paper evaluation. | benign_helpfulness_non_overrefusal:1.000claude-sonnet46-system-card | Lower is better | 0.16% | Benign helpfulness 1.6% |
| overall_overrefusal_rateclaude-sonnet46/overrefusal.csv:overall_overrefusal_rateMeasures overall overrefusal rate in the cited first-party or paper evaluation. | benign_helpfulness_non_overrefusal:1.000claude-sonnet46-system-card | Lower is better | 0.134% | Benign helpfulness 1.34% |
higher_difficulty_overrefusal_rate
Measures higher difficulty overrefusal rate in the cited first-party or paper evaluation.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-opus-4.6 | 0.04 | official | |
| 2 | claude-sonnet-4.6 | 0.18 | official | |
| 3 | claude-opus-4.5 | 0.83 | official | |
| 4 | claude-haiku-4.5 | 6.01 | official | |
| 5 | claude-sonnet-4.5 | 8.5 | official |
overall_overrefusal_rate
Measures overall overrefusal rate in the cited first-party or paper evaluation.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-sonnet-4.5 | 0.08 | official | |
| 2 | claude-sonnet-4.6 | 0.23 | official | |
| 3 | claude-haiku-4.5 | 0.26 | official | |
| 4 | claude-opus-4.6 | 0.66 | official | |
| 5 | claude-opus-4.5 | 0.8 | official |