← Evals

Evaluation profile

PandaBench JBB direct-request panel

1sub-evals
0.217%total index weight
1components

Within-component eval weight: Misuse resistance 2.17%.

Model score (higher is better)Predicted score

About this eval

Direct-request resistance on the 100-item JailbreakBench JBB-Behaviors instrument.

Included in the behavior ranking.

Sub-evals

MeasureComponentDirectionTotal index weightWithin-component weight
safety_ratepandabench/pandabench-jbb-direct.csv:safety_rateMeasures whether the model resists automated jailbreak attacks across one hundred harmful behaviors.ordinary_harm_misuse_resistance:1.000jailbreakbench-jbbHigher is better0.217%Misuse resistance 2.17%

safety_rate

Measures whether the model resists automated jailbreak attacks across one hundred harmful behaviors.

RankModelValueRelative performanceProvenance
1claude-3.7-sonnet1official
1doubao-lite1official
1gemini-2.0-flash1official
1gemini-2.0-flash-lite1official
1gemma-2-2b-it1official
1llama-3.1-tulu-3-70b1official
1llama-3.1-tulu-3-8b1official
1phi-3.5-moe-instruct1official
1qwen-2.5-1.5b-instruct1official
1qwen-2.5-14b-instruct1official
1qwen2-72b-instruct1official
1qwen2.5-32b-instruct1official
13gpt-4o0.995official
14claude-3.5-haiku0.99official
14claude-3.5-sonnet0.99official
14doubao-pro0.99official
14llama-3.2-1b-instruct0.99official
14llama-3.2-3b-instruct0.99official
14qwen-2.5-7b-instruct0.99official
14qwen2-7b-instruct0.99official
14qwen2.5-3b-instruct0.99official
14qwen2.5-72b-instruct0.99official
14qwen3-8b0.99official
24deepseek-v30.985official
25gpt-4o-mini0.98official
25llama-3.1-8b-instruct0.98official
25o3-mini0.98official
28deepseek-r1-distill-llama-70b0.97official
28gemini-2.0-pro0.97official
28glm-4-flash0.97official
28phi-3-mini-4k-instruct0.97official
28qwen3-14b0.97official
28qwen3-4b0.97official
34qwen3-30b-a3b0.96official
35doubao-1.5-lite0.95official
35doubao-1.5-pro0.95official
35glm-4-plus0.95official
35llama-3.1-70b-instruct0.95official
35llama-3.3-70b-instruct0.95official
40qwen-2.5-0.5b-instruct0.94official
40qwen3-32b0.94official
42deepseek-r10.93official
43llama-3.1-405b-instruct0.92official
44grok-20.86official
45qwen3-1.7b0.78official
46qwen3-0.6b0.62official