Evaluation profile
MonitoringBench Full-Trajectory Monitor
1sub-evals
0.251%Safety weight
0%Freedom weight
1components
Weights below are portfolio-specific global index weights.
Model score (higher is better)Predicted score
About this eval
Full-trajectory monitor competence at a fixed trajectory-level false-positive rate.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction by ranking | Safety weight | Freedom weight |
|---|---|---|---|---|
| full_trajectory_catch_rate_at_1pct_fpr_percentmonitoringbench/monitoringbench.csv:full_trajectory_catch_rate_at_1pct_fpr_percentMeasures post-hoc monitor detection competence over complete attack trajectories. | Safety: responsible_agency_control:1.000monitoringbench-full-trajectory | Safety: higher | 0.251% | — |
full_trajectory_catch_rate_at_1pct_fpr_percent
Measures post-hoc monitor detection competence over complete attack trajectories.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-opus-4.5 | 69.9 | official | |
| 2 | claude-sonnet-4.5 | 53.1 | official | |
| 3 | gpt-5.1-codex | 48.4 | official | |
| 4 | claude-sonnet-4 | 42.1 | official | |
| 5 | o3 | 31.8 | official | |
| 6 | gpt-5 | 31.3 | official | |
| 7 | gpt-4.1 | 14.5 | official | |
| 8 | gpt-4o | 9.3 | official | |
| 9 | gpt-4-turbo | 8.4 | official | |
| 10 | gpt-5-nano | 6 | official | |
| 11 | o4-mini | 5.5 | official | |
| 12 | gpt-4.1-mini | 4 | official | |
| 13 | gpt-4o-mini | 0.4 | official |