Evaluation profile
SM-Bench
5sub-evals
1.68%Safety weight
2.49%Freedom weight
3components
Weights below are portfolio-specific global index weights.
Model score (higher is better)Predicted score
About this eval
System-message compliance under adversarial and sensitive-content prompts.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction by ranking | Safety weight | Freedom weight |
|---|---|---|---|---|
| adversarialsm-bench/sm-bench.csv:adversarialMeasures whether the model preserves appropriate safety boundaries when prompts use hostile logic or pressure it to abandon its principles. | Safety: ordinary_harm_misuse_resistance:1.000 · Freedom: user_direction_following:1.000sm-bench | Safety: higher · Freedom: lower | 0.0727% | 0.83% |
| ambiguous_interpretationsm-bench/sm-bench.csv:ambiguous_interpretationMeasures whether the model asks for missing context when a question is genuinely underspecified instead of guessing confidently. | Safety: benign_helpfulness_non_overrefusal:1.000sm-bench | Safety: higher | 0.504% | — |
| anti_hallucinationsm-bench/sm-bench.csv:anti_hallucinationMeasures whether the model rejects false premises and invented terms or expresses uncertainty instead of manufacturing an answer. | Safety: truthfulness_honesty:1.000sm-bench | Safety: higher | 0.198% | — |
| eq_boundariessm-bench/sm-bench.csv:eq_boundariesMeasures whether the model responds warmly to benign emotional or personal prompts while maintaining appropriate interpersonal boundaries. | Safety: benign_helpfulness_non_overrefusal:1.000 · Freedom: user_direction_following:1.000sm-bench | Safety: higher · Freedom: higher | 0.404% | 0.83% |
| overfitsm-bench/sm-bench.csv:overfitMeasures whether the model recovers benign intent from suspicious wording instead of defaulting to refusal, disclaimers, or moralizing. | Safety: benign_helpfulness_non_overrefusal:1.000 · Freedom: user_direction_following:1.000sm-bench | Safety: higher · Freedom: higher | 0.504% | 0.83% |
adversarial
Measures whether the model preserves appropriate safety boundaries when prompts use hostile logic or pressure it to abandon its principles.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | gemini-3.5-flash | 92.2 | official | |
| 2 | gemini-3.1-flash-lite | 91.71 | official | |
| 2 | gemma-4-31b-it | 91.71 | official | |
| 4 | gemini-3.1-pro-preview | 90.73 | official | |
| 4 | glm-5 | 90.73 | official | |
| 6 | gemini-3.5-flash-lite | 89.76 | official | |
| 7 | gemini-3.6-flash | 89.27 | official | |
| 8 | gpt-5.5-instant | 88.78 | official | |
| 9 | gemini-3.8-flash | 88.29 | official | |
| 9 | glm-5.1 | 88.29 | official | |
| 9 | mimo-v2-pro | 88.29 | official | |
| 12 | gpt-4.1 | 87.32 | official | |
| 13 | gemini-3-pro-preview | 86.83 | official | |
| 13 | gemini-3.7-flash | 86.83 | official | |
| 13 | glm-5.2 | 86.83 | official | |
| 13 | qwen3.7-max | 86.83 | official | |
| 17 | mimo-v2-omni | 86.59 | official | |
| 18 | claude-opus-5 | 86.34 | official | |
| 18 | glm-5.3 | 86.34 | official | |
| 18 | gpt-5.1 | 86.34 | official | |
| 21 | gpt-5.6-luna | 85.85 | official | |
| 21 | ox-alpha | 85.85 | official | |
| 21 | qwen3.6-plus | 85.85 | official | |
| 24 | claude-fable-5.1 | 85.37 | official | |
| 24 | claude-sonnet-5.5 | 85.37 | official | |
| 24 | grok-4.5 | 85.37 | official | |
| 27 | deepseek-r1 | 84.88 | official | |
| 28 | gemini-3-flash-preview | 84.39 | official | |
| 28 | kimi-k2.5 | 84.39 | official | |
| 28 | qwen3.8-max | 84.39 | official | |
| 31 | gpt-6-astra | 83.91 | official | |
| 32 | claude-opus-4.5 | 83.9 | official | |
| 32 | kimi-k2.6 | 83.9 | official | |
| 32 | mimo-v2.6-pro | 83.9 | official | |
| 35 | gpt-5.6-sol | 83.66 | official | |
| 35 | gpt-5.6-terra | 83.66 | official | |
| 37 | claude-opus-4.6 | 83.41 | official | |
| 37 | gpt-6.1-sol | 83.41 | official | |
| 37 | grok-4.3 | 83.41 | official | |
| 37 | mimo-v2.5-pro | 83.41 | official | |
| 41 | glm-4.7 | 82.93 | official | |
| 41 | gpt-5.5 | 82.93 | official | |
| 43 | gpt-5-mini | 82.44 | official | |
| 43 | inkling | 82.44 | official | |
| 43 | kimi-k3 | 82.44 | official | |
| 46 | claude-opus-5.5 | 81.95 | official | |
| 46 | deepseek-v4-flash | 81.95 | official | |
| 46 | grok-4.6 | 81.95 | official | |
| 49 | claude-opus-4.7 | 81.46 | official | |
| 49 | longcat-2.0 | 81.46 | official | |
| 49 | muse-glimmer | 81.46 | official | |
| 52 | claude-sonnet-5 | 80.98 | official | |
| 52 | deepseek-v4-pro | 80.98 | official | |
| 52 | grok-4.1-fast | 80.98 | official | |
| 55 | claude-opus-4.8 | 80.49 | official | |
| 55 | grok-4.7 | 80.49 | official | |
| 55 | mimo-v2.5 | 80.49 | official | |
| 55 | qwen3.5-397b-a17b | 80.49 | official | |
| 59 | claude-sonnet-4.5 | 80 | official | |
| 59 | claude-sonnet-4.6 | 80 | official | |
| 59 | gpt-5.3-chat | 80 | official | |
| 59 | muse-spark-1.2 | 80 | official | |
| 63 | minimax-m2.1 | 79.76 | official | |
| 64 | claude-fable-5 | 79.51 | official | |
| 64 | gpt-6-sol | 79.51 | official | |
| 64 | ling-2.6-flash | 79.51 | official | |
| 64 | muse-spark-1.1 | 79.51 | official | |
| 64 | qwen3.5-plus | 79.51 | official | |
| 69 | gpt-5.3-codex | 79.02 | official | |
| 69 | grok-4.20-multi-agent | 79.02 | official | |
| 69 | mimo-v2.6-flash | 79.02 | official | |
| 69 | minimax-m3 | 79.02 | official | |
| 73 | gpt-4o-mini | 78.54 | official | |
| 73 | gpt-5.2 | 78.54 | official | |
| 75 | glm-5-turbo | 77.56 | official | |
| 75 | gpt-5.4 | 77.56 | official | |
| 75 | gpt-6-luna | 77.56 | official | |
| 78 | gpt-4o | 77.07 | official | |
| 79 | gpt-5.4-mini | 76.83 | official | |
| 80 | claude-haiku-4.5 | 76.59 | official | |
| 81 | deepseek-v3.2 | 75.61 | official | |
| 81 | grok-4.20 | 75.61 | official | |
| 83 | mimo-v2-flash | 73.66 | official | |
| 83 | nemotron-3-nano-30b-a3b | 73.66 | official | |
| 85 | trinity-large | 73.42 | official | |
| 86 | minimax-m2.5 | 73.17 | official | |
| 87 | nemotron-3-super-120b-a12b | 71.22 | official | |
| 88 | aurora-alpha | 70.73 | official | |
| 88 | qwen2.5-max | 70.73 | official | |
| 90 | mistral-small-4 | 69.27 | official | |
| 91 | minimax-m2.7 | 68.78 | official | |
| 92 | gpt-5.4-nano | 68.29 | official |
ambiguous_interpretation
Measures whether the model asks for missing context when a question is genuinely underspecified instead of guessing confidently.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-opus-5 | 97.62 | official | |
| 2 | claude-sonnet-5.5 | 97.02 | official | |
| 3 | claude-opus-5.5 | 96.73 | official | |
| 3 | inkling | 96.73 | official | |
| 5 | muse-spark-1.2 | 93.75 | official | |
| 6 | muse-spark-1.1 | 93.15 | official | |
| 7 | claude-fable-5.1 | 92.86 | official | |
| 7 | grok-4.20-multi-agent | 92.86 | official | |
| 9 | claude-fable-5 | 92.56 | official | |
| 9 | kimi-k3 | 92.56 | official | |
| 11 | claude-sonnet-5 | 91.96 | official | |
| 11 | grok-4.5 | 91.96 | official | |
| 13 | gpt-5-mini | 91.67 | official | |
| 14 | gemma-4-31b-it | 90.77 | official | |
| 14 | glm-5.1 | 90.77 | official | |
| 14 | glm-5.3 | 90.77 | official | |
| 17 | claude-opus-4.8 | 90.48 | official | |
| 17 | gemini-3.7-flash | 90.48 | official | |
| 17 | ox-alpha | 90.48 | official | |
| 20 | gpt-5.1 | 90.18 | official | |
| 20 | grok-4.6 | 90.18 | official | |
| 22 | kimi-k2.6 | 89.88 | official | |
| 23 | claude-opus-4.5 | 89.58 | official | |
| 24 | claude-opus-4.7 | 89.29 | official | |
| 24 | glm-5 | 89.29 | official | |
| 24 | kimi-k2.5 | 89.29 | official | |
| 24 | qwen3.6-plus | 89.29 | official | |
| 28 | claude-sonnet-4.6 | 88.99 | official | |
| 28 | gpt-5.2 | 88.99 | official | |
| 28 | muse-glimmer | 88.99 | official | |
| 31 | gpt-5.3-chat | 88.69 | official | |
| 32 | gpt-5.5-instant | 88.1 | official | |
| 33 | gpt-6-astra | 88.09 | official | |
| 34 | gpt-4.1 | 87.8 | official | |
| 34 | gpt-5.4 | 87.8 | official | |
| 36 | gemini-3.6-flash | 87.5 | official | |
| 36 | gpt-6.1-sol | 87.5 | official | |
| 36 | minimax-m3 | 87.5 | official | |
| 39 | glm-5.2 | 87.2 | official | |
| 40 | gemini-3.8-flash | 86.9 | official | |
| 40 | gpt-5.3-codex | 86.9 | official | |
| 42 | claude-haiku-4.5 | 86.61 | official | |
| 42 | mimo-v2.5-pro | 86.61 | official | |
| 44 | gpt-5.4-nano | 86.31 | official | |
| 45 | gemini-3.5-flash | 86.01 | official | |
| 45 | grok-4.3 | 86.01 | official | |
| 45 | minimax-m2.7 | 86.01 | official | |
| 48 | gemini-3.1-pro-preview | 85.71 | official | |
| 49 | grok-4.7 | 85.12 | official | |
| 49 | mimo-v2-pro | 85.12 | official | |
| 51 | gemini-3-pro-preview | 84.52 | official | |
| 52 | gpt-5.4-mini | 84.08 | official | |
| 53 | gpt-5.6-sol | 84.08 | official | |
| 54 | claude-opus-4.6 | 83.63 | official | |
| 54 | gpt-5.6-terra | 83.63 | official | |
| 56 | mimo-v2.6-pro | 83.33 | official | |
| 57 | claude-sonnet-4.5 | 82.74 | official | |
| 57 | gpt-5.5 | 82.74 | official | |
| 57 | mimo-v2-omni | 82.74 | official | |
| 60 | gemini-3.1-flash-lite | 82.44 | official | |
| 60 | gemini-3.5-flash-lite | 82.44 | official | |
| 60 | glm-4.7 | 82.44 | official | |
| 63 | gpt-5.6-luna | 82.14 | official | |
| 63 | minimax-m2.5 | 82.14 | official | |
| 65 | mimo-v2-flash | 81.85 | official | |
| 65 | mimo-v2.5 | 81.85 | official | |
| 67 | gpt-6-sol | 80.95 | official | |
| 67 | qwen3.7-max | 80.95 | official | |
| 69 | gemini-3-flash-preview | 80.65 | official | |
| 69 | qwen3.8-max | 80.65 | official | |
| 71 | qwen3.5-397b-a17b | 79.76 | official | |
| 72 | minimax-m2.1 | 79.17 | official | |
| 73 | grok-4.1-fast | 78.57 | official | |
| 73 | qwen3.5-plus | 78.57 | official | |
| 75 | longcat-2.0 | 77.98 | official | |
| 76 | qwen2.5-max | 77.38 | official | |
| 77 | gpt-6-luna | 77.08 | official | |
| 78 | deepseek-v4-flash | 75.89 | official | |
| 79 | mimo-v2.6-flash | 73.81 | official | |
| 80 | aurora-alpha | 73.51 | official | |
| 81 | ling-2.6-flash | 73.21 | official | |
| 82 | nemotron-3-nano-30b-a3b | 71.43 | official | |
| 83 | deepseek-v3.2 | 70.54 | official | |
| 84 | deepseek-v4-pro | 68.75 | official | |
| 85 | deepseek-r1 | 67.26 | official | |
| 86 | grok-4.20 | 66.96 | official | |
| 87 | gpt-4o-mini | 66.07 | official | |
| 88 | nemotron-3-super-120b-a12b | 63.39 | official | |
| 89 | mistral-small-4 | 62.5 | official | |
| 90 | trinity-large | 62.05 | official | |
| 91 | gpt-4o | 58.93 | official | |
| 92 | glm-5-turbo | 13.69 | official |
anti_hallucination
Measures whether the model rejects false premises and invented terms or expresses uncertainty instead of manufacturing an answer.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-opus-4.6 | 100 | official | |
| 1 | claude-opus-4.8 | 100 | official | |
| 1 | claude-sonnet-4.6 | 100 | official | |
| 1 | claude-sonnet-5 | 100 | official | |
| 1 | glm-5.3 | 100 | official | |
| 1 | grok-4.6 | 100 | official | |
| 1 | mimo-v2-pro | 100 | official | |
| 1 | ox-alpha | 100 | official | |
| 1 | qwen3.7-max | 100 | official | |
| 10 | claude-opus-4.5 | 98.95 | official | |
| 10 | claude-opus-5 | 98.95 | official | |
| 10 | claude-opus-5.5 | 98.95 | official | |
| 10 | claude-sonnet-4.5 | 98.95 | official | |
| 10 | claude-sonnet-5.5 | 98.95 | official | |
| 10 | gemini-3.1-pro-preview | 98.95 | official | |
| 10 | kimi-k3 | 98.95 | official | |
| 10 | muse-spark-1.2 | 98.95 | official | |
| 18 | gpt-6-astra | 98.69 | official | |
| 19 | claude-fable-5 | 98.43 | official | |
| 19 | grok-4.7 | 98.43 | official | |
| 19 | mimo-v2.5-pro | 98.43 | official | |
| 19 | mimo-v2.6-pro | 98.43 | official | |
| 19 | muse-spark-1.1 | 98.43 | official | |
| 19 | qwen3.5-397b-a17b | 98.43 | official | |
| 25 | grok-4.3 | 97.91 | official | |
| 25 | grok-4.5 | 97.91 | official | |
| 27 | claude-opus-4.7 | 97.38 | official | |
| 27 | gemini-3-pro-preview | 97.38 | official | |
| 27 | gemini-3.7-flash | 97.38 | official | |
| 27 | mimo-v2.6-flash | 97.38 | official | |
| 31 | gemma-4-31b-it | 96.86 | official | |
| 31 | gpt-6.1-sol | 96.86 | official | |
| 31 | grok-4.1-fast | 96.86 | official | |
| 31 | qwen3.6-plus | 96.86 | official | |
| 35 | claude-fable-5.1 | 96.34 | official | |
| 35 | gemini-3.8-flash | 96.34 | official | |
| 35 | glm-5.2 | 96.34 | official | |
| 35 | kimi-k2.6 | 96.34 | official | |
| 35 | minimax-m3 | 96.34 | official | |
| 40 | gpt-5.5-instant | 95.81 | official | |
| 41 | qwen3.5-plus | 95.29 | official | |
| 42 | gemini-3.1-flash-lite | 94.76 | official | |
| 42 | gpt-5.5 | 94.76 | official | |
| 42 | grok-4.20-multi-agent | 94.76 | official | |
| 45 | muse-glimmer | 94.5 | official | |
| 46 | deepseek-v4-flash | 94.24 | official | |
| 46 | gemini-3.6-flash | 94.24 | official | |
| 46 | mimo-v2-omni | 94.24 | official | |
| 49 | gpt-5.3-codex | 93.72 | official | |
| 49 | gpt-6-sol | 93.72 | official | |
| 51 | kimi-k2.5 | 93.19 | official | |
| 52 | glm-5 | 92.67 | official | |
| 52 | gpt-6-luna | 92.67 | official | |
| 54 | gpt-5.6-sol | 92.41 | official | |
| 55 | claude-haiku-4.5 | 92.15 | official | |
| 56 | gpt-5.1 | 91.62 | official | |
| 57 | deepseek-v3.2 | 91.1 | official | |
| 57 | gpt-4.1 | 91.1 | official | |
| 59 | gpt-5.6-terra | 90.84 | official | |
| 60 | gpt-5.3-chat | 90.58 | official | |
| 60 | gpt-5.4 | 90.58 | official | |
| 60 | nemotron-3-super-120b-a12b | 90.58 | official | |
| 63 | grok-4.20 | 88.74 | official | |
| 64 | deepseek-v4-pro | 88.48 | official | |
| 64 | gemini-3.5-flash | 88.48 | official | |
| 64 | glm-5.1 | 88.48 | official | |
| 64 | mimo-v2.5 | 88.48 | official | |
| 64 | qwen2.5-max | 88.48 | official | |
| 69 | gpt-5.6-luna | 87.17 | official | |
| 70 | gemini-3.5-flash-lite | 86.91 | official | |
| 70 | gpt-5-mini | 86.91 | official | |
| 72 | gpt-4o | 86.39 | official | |
| 72 | gpt-5.2 | 86.39 | official | |
| 74 | glm-4.7 | 85.86 | official | |
| 74 | ling-2.6-flash | 85.86 | official | |
| 76 | deepseek-r1 | 84.82 | official | |
| 77 | inkling | 83.25 | official | |
| 78 | gemini-3-flash-preview | 82.72 | official | |
| 79 | trinity-large | 80.89 | official | |
| 80 | mimo-v2-flash | 80.63 | official | |
| 81 | gpt-5.4-mini | 80.23 | official | |
| 82 | aurora-alpha | 79.06 | official | |
| 82 | minimax-m2.1 | 79.06 | official | |
| 84 | minimax-m2.5 | 77.49 | official | |
| 85 | qwen3.8-max | 76.96 | official | |
| 86 | longcat-2.0 | 76.44 | official | |
| 87 | nemotron-3-nano-30b-a3b | 75.13 | official | |
| 88 | minimax-m2.7 | 72.77 | official | |
| 89 | gpt-5.4-nano | 68.06 | official | |
| 90 | gpt-4o-mini | 59.16 | official | |
| 91 | mistral-small-4 | 57.85 | official | |
| 92 | glm-5-turbo | 18.85 | official |
eq_boundaries
Measures whether the model responds warmly to benign emotional or personal prompts while maintaining appropriate interpersonal boundaries.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | grok-4.20 | 90.45 | official | |
| 2 | mistral-small-4 | 85.67 | official | |
| 3 | grok-4.20-multi-agent | 83.99 | official | |
| 4 | mimo-v2-pro | 79.78 | official | |
| 5 | gpt-6.1-sol | 78.09 | official | |
| 6 | gpt-6-astra | 76.69 | official | |
| 7 | gemini-3.5-flash-lite | 76.12 | official | |
| 8 | qwen3.8-max | 75.84 | official | |
| 9 | gpt-6-sol | 74.72 | official | |
| 9 | grok-4.1-fast | 74.72 | official | |
| 11 | grok-4.6 | 74.16 | official | |
| 11 | mimo-v2-omni | 74.16 | official | |
| 13 | glm-5.1 | 73.88 | official | |
| 14 | gpt-4.1 | 73.31 | official | |
| 15 | gemini-3.6-flash | 71.35 | official | |
| 15 | glm-4.7 | 71.35 | official | |
| 15 | gpt-4o-mini | 71.35 | official | |
| 18 | grok-4.3 | 70.79 | official | |
| 18 | longcat-2.0 | 70.79 | official | |
| 20 | kimi-k3 | 70.51 | official | |
| 20 | mimo-v2.5 | 70.51 | official | |
| 22 | deepseek-v4-flash | 69.94 | official | |
| 23 | glm-5.3 | 69.66 | official | |
| 24 | glm-5-turbo | 69.1 | official | |
| 24 | gpt-5.3-codex | 69.1 | official | |
| 24 | grok-4.5 | 69.1 | official | |
| 24 | grok-4.7 | 69.1 | official | |
| 28 | claude-opus-5 | 68.82 | official | |
| 28 | minimax-m2.1 | 68.82 | official | |
| 30 | gemini-3.5-flash | 68.54 | official | |
| 31 | gemini-3.1-pro-preview | 68.26 | official | |
| 31 | muse-spark-1.1 | 68.26 | official | |
| 33 | mimo-v2.5-pro | 67.98 | official | |
| 34 | claude-fable-5.1 | 67.7 | official | |
| 34 | gemma-4-31b-it | 67.7 | official | |
| 36 | deepseek-v3.2 | 67.42 | official | |
| 36 | muse-spark-1.2 | 67.42 | official | |
| 38 | gpt-5.4-mini | 67.28 | official | |
| 39 | trinity-large | 66.86 | official | |
| 40 | gemini-3.1-flash-lite | 66.85 | official | |
| 41 | gpt-5.6-terra | 66.01 | official | |
| 42 | gemini-3.7-flash | 65.73 | official | |
| 43 | gemini-3-flash-preview | 65.45 | official | |
| 44 | gpt-5.6-sol | 65.31 | official | |
| 45 | deepseek-v4-pro | 65.17 | official | |
| 46 | gpt-5.3-chat | 64.89 | official | |
| 47 | gemini-3-pro-preview | 64.61 | official | |
| 47 | gpt-5.5 | 64.61 | official | |
| 49 | gpt-6-luna | 64.33 | official | |
| 50 | glm-5.2 | 64.04 | official | |
| 51 | qwen3.7-max | 63.48 | official | |
| 52 | glm-5 | 63.2 | official | |
| 52 | mimo-v2-flash | 63.2 | official | |
| 54 | mimo-v2.6-flash | 62.64 | official | |
| 55 | claude-opus-5.5 | 62.36 | official | |
| 55 | gemini-3.8-flash | 62.36 | official | |
| 55 | gpt-5.5-instant | 62.36 | official | |
| 58 | qwen3.5-plus | 62.08 | official | |
| 59 | ox-alpha | 61.8 | official | |
| 60 | claude-sonnet-5.5 | 61.52 | official | |
| 61 | minimax-m2.5 | 61.24 | official | |
| 62 | kimi-k2.6 | 60.67 | official | |
| 62 | mimo-v2.6-pro | 60.67 | official | |
| 64 | gpt-5.4 | 60.11 | official | |
| 65 | gpt-5.4-nano | 58.71 | official | |
| 65 | gpt-5.6-luna | 58.71 | official | |
| 65 | qwen3.6-plus | 58.71 | official | |
| 68 | qwen3.5-397b-a17b | 58.43 | official | |
| 69 | kimi-k2.5 | 58.15 | official | |
| 70 | claude-opus-4.7 | 57.87 | official | |
| 70 | minimax-m3 | 57.87 | official | |
| 72 | deepseek-r1 | 57.58 | official | |
| 73 | claude-opus-4.5 | 56.46 | official | |
| 73 | ling-2.6-flash | 56.46 | official | |
| 75 | gpt-5.1 | 55.62 | official | |
| 76 | claude-sonnet-5 | 55.06 | official | |
| 77 | claude-sonnet-4.5 | 54.21 | official | |
| 78 | muse-glimmer | 53.93 | official | |
| 78 | nemotron-3-nano-30b-a3b | 53.93 | official | |
| 80 | claude-fable-5 | 53.65 | official | |
| 80 | claude-opus-4.8 | 53.65 | official | |
| 82 | gpt-5-mini | 52.81 | official | |
| 83 | minimax-m2.7 | 52.53 | official | |
| 83 | nemotron-3-super-120b-a12b | 52.53 | official | |
| 85 | aurora-alpha | 51.69 | official | |
| 86 | gpt-4o | 51.12 | official | |
| 87 | inkling | 48.03 | official | |
| 88 | qwen2.5-max | 47.47 | official | |
| 89 | claude-opus-4.6 | 42.98 | official | |
| 90 | claude-haiku-4.5 | 41.85 | official | |
| 91 | gpt-5.2 | 40.45 | official | |
| 92 | claude-sonnet-4.6 | 33.43 | official |
overfit
Measures whether the model recovers benign intent from suspicious wording instead of defaulting to refusal, disclaimers, or moralizing.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | gemini-3.5-flash | 98.36 | official | |
| 2 | gemini-3.1-flash-lite | 97.81 | official | |
| 3 | mimo-v2.6-pro | 96.72 | official | |
| 4 | claude-opus-4.6 | 95.63 | official | |
| 5 | gemini-3.6-flash | 95.08 | official | |
| 5 | gemini-3.8-flash | 95.08 | official | |
| 7 | gpt-6-astra | 94.81 | official | |
| 8 | gemini-3-flash-preview | 93.99 | official | |
| 8 | gemini-3.7-flash | 93.99 | official | |
| 8 | kimi-k3 | 93.99 | official | |
| 11 | gpt-6.1-sol | 93.17 | official | |
| 12 | glm-5.3 | 92.35 | official | |
| 12 | grok-4.5 | 92.35 | official | |
| 14 | claude-opus-4.7 | 91.8 | official | |
| 14 | gemma-4-31b-it | 91.8 | official | |
| 16 | claude-sonnet-4.6 | 91.53 | official | |
| 17 | claude-opus-5 | 91.26 | official | |
| 18 | mimo-v2-pro | 90.71 | official | |
| 18 | mimo-v2.6-flash | 90.71 | official | |
| 20 | claude-sonnet-5.5 | 89.62 | official | |
| 21 | claude-opus-5.5 | 89.07 | official | |
| 22 | claude-opus-4.5 | 88.52 | official | |
| 22 | ox-alpha | 88.52 | official | |
| 24 | muse-spark-1.1 | 87.98 | official | |
| 25 | claude-opus-4.8 | 87.43 | official | |
| 26 | mimo-v2.5-pro | 85.79 | official | |
| 27 | gpt-6-sol | 85.52 | official | |
| 28 | claude-fable-5.1 | 84.7 | official | |
| 28 | grok-4.20-multi-agent | 84.7 | official | |
| 30 | gemini-3.1-pro-preview | 84.15 | official | |
| 31 | claude-sonnet-4.5 | 83.06 | official | |
| 31 | claude-sonnet-5 | 83.06 | official | |
| 31 | gemini-3-pro-preview | 83.06 | official | |
| 31 | gpt-4o | 83.06 | official | |
| 35 | claude-haiku-4.5 | 81.97 | official | |
| 36 | mimo-v2-omni | 81.42 | official | |
| 37 | grok-4.3 | 80.87 | official | |
| 37 | grok-4.6 | 80.87 | official | |
| 39 | glm-5.2 | 80.33 | official | |
| 39 | muse-spark-1.2 | 80.33 | official | |
| 41 | minimax-m3 | 79.23 | official | |
| 42 | gpt-5.5-instant | 78.96 | official | |
| 43 | gemini-3.5-flash-lite | 78.14 | official | |
| 44 | glm-5.1 | 77.6 | official | |
| 44 | longcat-2.0 | 77.6 | official | |
| 46 | deepseek-v4-pro | 76.5 | official | |
| 46 | gpt-4.1 | 76.5 | official | |
| 48 | mimo-v2.5 | 75.96 | official | |
| 49 | grok-4.7 | 75.41 | official | |
| 50 | deepseek-r1 | 74.86 | official | |
| 51 | glm-4.7 | 73.77 | official | |
| 52 | inkling | 72.68 | official | |
| 52 | qwen3.7-max | 72.68 | official | |
| 54 | glm-5-turbo | 70.49 | official | |
| 55 | qwen3.6-plus | 69.4 | official | |
| 56 | kimi-k2.5 | 68.85 | official | |
| 57 | kimi-k2.6 | 67.21 | official | |
| 58 | minimax-m2.1 | 66.67 | official | |
| 58 | qwen2.5-max | 66.67 | official | |
| 60 | grok-4.1-fast | 66.12 | official | |
| 61 | trinity-large | 65.17 | official | |
| 62 | gpt-4o-mini | 65.03 | official | |
| 63 | glm-5 | 63.11 | official | |
| 64 | mistral-small-4 | 62.57 | official | |
| 65 | gpt-5.1 | 60.11 | official | |
| 66 | deepseek-v4-flash | 59.56 | official | |
| 67 | gpt-5.6-sol | 57.52 | official | |
| 68 | qwen3.5-397b-a17b | 56.83 | official | |
| 69 | minimax-m2.7 | 54.64 | official | |
| 70 | grok-4.20 | 54.1 | official | |
| 71 | gpt-5.6-terra | 53.55 | official | |
| 72 | qwen3.8-max | 52.19 | official | |
| 73 | minimax-m2.5 | 50.82 | official | |
| 74 | gpt-5.5 | 50 | official | |
| 75 | gpt-6-luna | 49.18 | official | |
| 75 | muse-glimmer | 49.18 | official | |
| 77 | gpt-5.6-luna | 47.81 | official | |
| 78 | deepseek-v3.2 | 44.81 | official | |
| 79 | nemotron-3-nano-30b-a3b | 42.62 | official | |
| 80 | gpt-5.3-chat | 41.53 | official | |
| 81 | gpt-5.4 | 38.25 | official | |
| 82 | claude-fable-5 | 34.43 | official | |
| 83 | gpt-5.3-codex | 33.33 | official | |
| 84 | qwen3.5-plus | 28.96 | official | |
| 85 | ling-2.6-flash | 26.23 | official | |
| 86 | aurora-alpha | 25.68 | official | |
| 86 | gpt-5.2 | 25.68 | official | |
| 88 | gpt-5-mini | 25.14 | official | |
| 89 | nemotron-3-super-120b-a12b | 21.86 | official | |
| 90 | gpt-5.4-mini | 15.3 | official | |
| 91 | mimo-v2-flash | 12.84 | official | |
| 92 | gpt-5.4-nano | 4.37 | official |