Evaluation profile
VETO Misfired Alignment
1sub-evals
1.93%total index weight
2components
Within-component eval weight: Fairness 10.3% · Benign helpfulness 9%.
Model score (lower is better)Predicted score
About this eval
VETO Misfired Alignment
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction | Total index weight | Within-component weight |
|---|---|---|---|---|
| misfired_alignment_rate_pctveto/veto.csv:misfired_alignment_rate_pctMeasures misfired alignment rate pct in the cited first-party or paper evaluation. | fairness_nondiscrimination:0.650|benign_helpfulness_non_overrefusal:0.350veto-misfired-alignment | Lower is better | 1.93% | Fairness 10.3% · Benign helpfulness 9% |
misfired_alignment_rate_pct
Measures misfired alignment rate pct in the cited first-party or paper evaluation.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | deepseek-r1 | 4.7 | official | |
| 2 | gemini-3.1-pro-preview | 4.9 | official | |
| 3 | deepseek-v3 | 5.2 | official | |
| 4 | qwen3.5-27b | 5.5 | official | |
| 4 | qwen3.5-9b | 5.5 | official | |
| 6 | mistral-7b-instruct | 5.7 | official | |
| 7 | llama-3.1-8b-instruct | 6.2 | official | |
| 8 | gemma-3-27b-it | 6.3 | official | |
| 9 | gpt-5.5 | 7.5 | official | |
| 10 | grok-4.20 | 7.9 | official | |
| 10 | qwen3-4b | 7.9 | official | |
| 10 | qwen3.5-4b | 7.9 | official | |
| 13 | gemini-3.1-flash-lite | 8.3 | official | |
| 14 | llama-3.1-70b-instruct | 8.6 | official | |
| 15 | qwen3-32b | 8.7 | official | |
| 16 | qwen2.5-7b | 8.9 | official | |
| 17 | gpt-5.4-mini | 9.9 | official | |
| 18 | qwen3-14b | 10.2 | official | |
| 19 | qwen3-8b | 10.3 | official | |
| 20 | claude-opus-4.7 | 10.7 | official | |
| 21 | claude-sonnet-4.6 | 10.9 | official | |
| 22 | qwen2.5-72b-instruct | 11.4 | official | |
| 23 | llama-3.2-3b | 11.8 | official | |
| 24 | gpt-5.4 | 17.6 | official | |
| 25 | gpt-5.4-nano | 18.9 | official |