Evaluation profile
AutoElicit Transferability
1sub-evals
0.391%total index weight
1components
Within-component eval weight: Responsible agency 2.6%.
Model score (lower is better)Predicted score
About this eval
Unintended computer-use behavior under benign perturbations
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction | Total index weight | Within-component weight |
|---|---|---|---|---|
| elicitation_rateautoelicit/autoelicit.csv:elicitation_rateMeasures benign perturbations eliciting unintended computer-use behavior. | responsible_agency_control:1.000autoelicit | Lower is better | 0.391% | Responsible agency 2.6% |
elicitation_rate
Measures benign perturbations eliciting unintended computer-use behavior.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | evocua-32b | 35 | official | |
| 2 | evocua-8b | 37.6 | official | |
| 3 | claude-sonnet-4.5 | 41 | official | |
| 4 | opencua-32b | 43.6 | official | |
| 5 | opencua-7b | 47 | official | |
| 6 | operator | 48.7 | official | |
| 7 | opencua-72b | 53.8 | official |