Evaluation profile

BioSecBench-Refusal V2

2sub-evals

0.177%Safety weight

0.636%Freedom weight

2components

Weights below are portfolio-specific global index weights.

Model score (higher is better)Predicted score

About this eval

Deployed-agent refusal of concealed biological hazards and non-overrefusal of legitimate biological work.

Included in the behavior ranking.

Measure	Component	Direction by ranking	Safety weight	Freedom weight
redteam_refusal_pctbenchmarks-bio-security/benchmarks-bio-security.csv:redteam_refusal_pctMeasures whether the deployed agent refuses requests concealing biological hazards.	Safety: ordinary_harm_misuse_resistance:1.000 · Freedom: user_direction_following:1.000biosecbench-refusal	Safety: higher · Freedom: lower	0.0223%	0.318%
routine_compliance_pctbenchmarks-bio-security/benchmarks-bio-security.csv:routine_compliance_pctMeasures whether the deployed agent allows legitimate biological research requests without over-refusing them.	Safety: benign_helpfulness_non_overrefusal:1.000 · Freedom: user_direction_following:1.000biosecbench-refusal	Safety: higher · Freedom: higher	0.155%	0.318%

Measures whether the deployed agent refuses requests concealing biological hazards.

Rank	Model	Value	Provenance
1	gpt-6-astra	96.01	official
2	claude-opus-5	91.67	official
3	grok-4.7	79.71	official
4	gemini-3.8-flash	77.54	official
5	gemini-3.7-flash	63.64	official
6	grok-4.6	42.39	official

Measures whether the deployed agent allows legitimate biological research requests without over-refusing them.

Rank	Model	Value	Provenance
1	grok-4.7	53.59	official
2	grok-4.6	53.57	official
3	gemini-3.7-flash	49.72	official
4	gemini-3.8-flash	36.61	official
5	gpt-6-astra	22.4	official
6	claude-opus-5	18.58	official