← Models

Model profile

Claude 2

Anthropicdeveloper
2023-07-11release date
#39 / 346Safety rank
#625 / 662Freedom rank

Evidence summary

Safety. Claude 2 has an estimated Safety rank of #39; its 90% source-sensitivity interval is #5–#169. Its behavior-only rank is #48; company governance moves the combined estimate to #39. Published Safety evidence spans 9 eval lineages and 5 of 7 components. Its strongest relative result is SALAD-Bench (base_representation_toxicity, #1 of 33); its weakest is Claude 3 model-card adversarial human-preference evaluations (human_feedback_harmlessness_win_rate_pct, #4 of 4).

Freedom. Claude 2 has an estimated Freedom rank of #625; its 90% source-sensitivity interval is #402–#650. Published Freedom evidence spans 9 eval lineages and 1 of 1 components. Its strongest relative result is Human Pathogen Capabilities Test (HPCT) — overall refusal (refusal_rate_pct, #1 of 69); its weakest is Claude 2 model-card safety and alignment evaluations (red_teaming_rank, #3 of 3).

Compare this model

Only models sharing at least one published sub-eval are listed.

Official and reference links

Safety evals

Rank and direction are specific to the Safety portfolio. Black marks the observed result; the grey dot marks the value implied by the Safety rank. Values stay on each source’s native scale.

Eval / sub-evalRankValueDistributionSource
Claude 2 model-card safety and alignment evaluationshhh#1 / 3↑0.86Source ↗official
Claude 2 model-card safety and alignment evaluationshuman_feedback_harmless_elo#2 / 3↑278Source ↗official
Claude 2 model-card safety and alignment evaluationshuman_feedback_honest_elo#1 / 3↑137Source ↗official
Claude 2 model-card safety and alignment evaluationsred_teaming_rank#1 / 3↓1Source ↗official
Claude 3 model-card adversarial human-preference evaluationscorrect_refusals_wildchat_rank#1 / 5↓1Source ↗official
Claude 3 model-card adversarial human-preference evaluationshuman_feedback_harmlessness_win_rate_pct#4 / 4↑84Source ↗official
Claude 3 model-card adversarial human-preference evaluationshuman_feedback_honesty_win_rate_pct#3 / 4↑66Source ↗official
Claude 3 model-card adversarial human-preference evaluationsincorrect_refusals_wildchat_rank#4 / 5↓4Source ↗official
Claude 3 model-card adversarial human-preference evaluationsincorrect_refusals_xstest_rank#2 / 5↓2Source ↗official
DecodingTrustmachine_ethics#3 / 8↑85.17Source ↗official
DecodingTruststereotype_bias#1 / 8↑100Source ↗official
DecodingTrusttoxicity#1 / 8↑92.11Source ↗official
Fake Alignment (FINE)multiple_choice_safe_decision_rate#2 / 14↑85.33Source ↗official
Fake Alignment (FINE)open_ended_safe_response_rate#3 / 14↑98.67Source ↗official
HarmBenchdr#2 / 28↓2Source ↗official
Human Pathogen Capabilities Test (HPCT) — overall refusalrefusal_rate_pct#27 / 69↑0Source ↗official
SALAD-Benchattack_enhanced_human_autonomy_integrity#1 / 33↑86.64Source ↗official
SALAD-Benchattack_enhanced_information_safety_harms#1 / 33↑93.49Source ↗official
SALAD-Benchattack_enhanced_malicious_use#1 / 33↑87.03Source ↗official
SALAD-Benchattack_enhanced_misinformation_harms#1 / 33↑91.61Source ↗official
SALAD-Benchattack_enhanced_representation_toxicity#1 / 33↑87.15Source ↗official
SALAD-Benchattack_enhanced_socioeconomic_harms#1 / 33↑88.31Source ↗official
SALAD-Benchbase_human_autonomy_integrity#1 / 33↑99.88Source ↗official
SALAD-Benchbase_information_safety_harms#2 / 33↑99.66Source ↗official
SALAD-Benchbase_malicious_use#1 / 33↑99.97Source ↗official
SALAD-Benchbase_misinformation_harms#1 / 33↑99.7Source ↗official
SALAD-Benchbase_representation_toxicity#1 / 33↑99.58Source ↗official
SALAD-Benchbase_socioeconomic_harms#1 / 33↑99.41Source ↗official
SALAD-Benchmcq_human_autonomy_integrity#22 / 33↑21.67Source ↗official
SALAD-Benchmcq_information_safety_harms#20 / 33↑28.61Source ↗official
SALAD-Benchmcq_malicious_use#23 / 33↑19.17Source ↗official
SALAD-Benchmcq_misinformation_harms#23 / 33↑20.71Source ↗official
SALAD-Benchmcq_representation_toxicity#21 / 33↑24.38Source ↗official
SALAD-Benchmcq_socioeconomic_harms#20 / 33↑29.44Source ↗official
SORRY-Benchavg#2 / 51↓0.07Source ↗official
SuperCLUE Safetyinstruction_attack#8 / 31↑70.69Source ↗official
SuperCLUE Safetyresponsible_ai#1 / 31↑78.18Source ↗official
SuperCLUE Safetytraditional_safety#14 / 31↑77.66Source ↗official

Freedom evals

Rank and direction are specific to the Freedom portfolio. Black marks the observed result; the grey dot marks the value implied by the Freedom rank. Values stay on each source’s native scale.

Eval / sub-evalRankValueDistributionSource
Claude 2 model-card safety and alignment evaluationshuman_feedback_harmless_elo#2 / 3↓278Source ↗official
Claude 2 model-card safety and alignment evaluationsred_teaming_rank#3 / 3↑1Source ↗official
Claude 3 model-card adversarial human-preference evaluationscorrect_refusals_wildchat_rank#5 / 5↑1Source ↗official
Claude 3 model-card adversarial human-preference evaluationshuman_feedback_harmlessness_win_rate_pct#1 / 4↓84Source ↗official
Claude 3 model-card adversarial human-preference evaluationsincorrect_refusals_wildchat_rank#4 / 5↓4Source ↗official
Claude 3 model-card adversarial human-preference evaluationsincorrect_refusals_xstest_rank#2 / 5↓2Source ↗official
DecodingTrusttoxicity#8 / 8↓92.11Source ↗official
Fake Alignment (FINE)open_ended_safe_response_rate#11 / 14↓98.67Source ↗official
HarmBenchdr#26 / 28↑2Source ↗official
Human Pathogen Capabilities Test (HPCT) — overall refusalrefusal_rate_pct#1 / 69↓0Source ↗official
SALAD-Benchattack_enhanced_human_autonomy_integrity#33 / 33↓86.64Source ↗official
SALAD-Benchattack_enhanced_information_safety_harms#33 / 33↓93.49Source ↗official
SALAD-Benchattack_enhanced_malicious_use#33 / 33↓87.03Source ↗official
SALAD-Benchattack_enhanced_misinformation_harms#33 / 33↓91.61Source ↗official
SALAD-Benchattack_enhanced_representation_toxicity#33 / 33↓87.15Source ↗official
SALAD-Benchattack_enhanced_socioeconomic_harms#33 / 33↓88.31Source ↗official
SALAD-Benchbase_human_autonomy_integrity#33 / 33↓99.88Source ↗official
SALAD-Benchbase_information_safety_harms#32 / 33↓99.66Source ↗official
SALAD-Benchbase_malicious_use#33 / 33↓99.97Source ↗official
SALAD-Benchbase_misinformation_harms#33 / 33↓99.7Source ↗official
SALAD-Benchbase_representation_toxicity#33 / 33↓99.58Source ↗official
SALAD-Benchbase_socioeconomic_harms#33 / 33↓99.41Source ↗official
SORRY-Benchavg#48 / 51↑0.07Source ↗official
SuperCLUE Safetyinstruction_attack#24 / 31↓70.69Source ↗official
SuperCLUE Safetytraditional_safety#18 / 31↓77.66Source ↗official