← Models

Model profile

Llama 2 7B Chat

Metadeveloper
2023-07-18release date
#231 / 346Safety rank
#584 / 662Freedom rank

Evidence summary

Safety. Llama 2 7B Chat has an estimated Safety rank of #231; its 90% source-sensitivity interval is #140–#268. Its behavior-only rank is #218; company governance moves the combined estimate to #231. Published Safety evidence spans 16 eval lineages and 7 of 7 components. Its strongest relative result is MedSafetyBench (medical_safety_score, #1 of 30); its weakest is NESSiE Necessary Safety Benchmark (sh_score, #16 of 16).

Freedom. Llama 2 7B Chat has an estimated Freedom rank of #584; its 90% source-sensitivity interval is #301–#633. Published Freedom evidence spans 11 eval lineages and 1 of 1 components. Its strongest relative result is COMPL-AI LLM RuLES Multi-Turn Rule Following (score, #3 of 14); its weakest is Do-Not-Answer (human_harmlessness_rate, #6 of 6).

Compare this model

Only models sharing at least one published sub-eval are listed.

Official and reference links

Safety evals

Rank and direction are specific to the Safety portfolio. Black marks the observed result; the grey dot marks the value implied by the Safety rank. Values stay on each source’s native scale.

Eval / sub-evalRankValueDistributionSource
COMPL-AI AI-Identity Disclosurescore#8 / 14↑0.9315Source ↗official
COMPL-AI LLM RuLES Multi-Turn Rule Followingscore#12 / 14↑0.2699Source ↗official
COMPL-AI TensorTrust Goal-Hijacking Resistancescore#5 / 13↑0.5142Source ↗official
Contextual MoralChoicehuman_agreement#21 / 22↑0.3Source ↗official
DecodingTrustmachine_ethics#7 / 8↑40.58Source ↗official
DecodingTruststereotype_bias#3 / 8↑97.6Source ↗official
DecodingTrusttoxicity#2 / 8↑80Source ↗official
Do-Not-Answerhuman_harmlessness_rate#1 / 6↑99.68Source ↗official
Enkrypt AI Safety Leaderboardbias_attack_non_success_rate#111 / 270↑18.6Source ↗official
Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate#90 / 270↑89.83Source ↗official
Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate#92 / 270↑85.56Source ↗official
Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate#23 / 268↑99.5Source ↗official
HarmBenchdr#1 / 28↓0.8Source ↗official
IndoBias-Pairs — parity-aware culturally grounded biasparity_score#4 / 26↑89.73Source ↗official
JailBenchjailbreak_success_rate#3 / 14↓48.68Source ↗official
Large-scale Moral Machine experiment on LLMshuman_choice_distance#20 / 39↓1.077Source ↗official
MedSafetyBenchmedical_safety_score#1 / 30↑99.81Source ↗official
NESSiE Necessary Safety Benchmarksh_score#16 / 16↑15.38Source ↗official
OR-Benchover_refusal_rate#18 / 25↓87.5Source ↗official
OR-Benchtoxic_acceptance_rate#6 / 25↓0.4Source ↗official
S-Evalbase_en_overall#3 / 22↑78.2Source ↗official
SafetyBenchEM#20 / 21↑49.8Source ↗official
SafetyBenchIA#19 / 21↑62.4Source ↗official
SafetyBenchMH#18 / 21↑70.2Source ↗official
SafetyBenchOFF#18 / 21↑48.9Source ↗official
SafetyBenchPH#18 / 21↑54.5Source ↗official
SafetyBenchPP#18 / 21↑65Source ↗official
SafetyBenchUB#9 / 21↑63.2Source ↗official
SALAD-Benchattack_enhanced_human_autonomy_integrity#13 / 33↑20.47Source ↗official
SALAD-Benchattack_enhanced_information_safety_harms#14 / 33↑17.59Source ↗official
SALAD-Benchattack_enhanced_malicious_use#14 / 33↑16.31Source ↗official
SALAD-Benchattack_enhanced_misinformation_harms#13 / 33↑18.75Source ↗official
SALAD-Benchattack_enhanced_representation_toxicity#14 / 33↑18.21Source ↗official
SALAD-Benchattack_enhanced_socioeconomic_harms#8 / 33↑23.81Source ↗official
SALAD-Benchbase_human_autonomy_integrity#5 / 33↑98.37Source ↗official
SALAD-Benchbase_information_safety_harms#3 / 33↑99.26Source ↗official
SALAD-Benchbase_malicious_use#4 / 33↑98.54Source ↗official
SALAD-Benchbase_misinformation_harms#9 / 33↑96.26Source ↗official
SALAD-Benchbase_representation_toxicity#9 / 33↑93.06Source ↗official
SALAD-Benchbase_socioeconomic_harms#6 / 33↑94.12Source ↗official
SALAD-Benchmcq_human_autonomy_integrity#27 / 33↑6.389Source ↗official
SALAD-Benchmcq_information_safety_harms#29 / 33↑3.333Source ↗official
SALAD-Benchmcq_malicious_use#28 / 33↑4.551Source ↗official
SALAD-Benchmcq_misinformation_harms#27 / 33↑6.19Source ↗official
SALAD-Benchmcq_representation_toxicity#28 / 33↑4.271Source ↗official
SALAD-Benchmcq_socioeconomic_harms#29 / 33↑1.667Source ↗official
SORRY-Benchavg#12 / 51↓0.14Source ↗official

Freedom evals

Rank and direction are specific to the Freedom portfolio. Black marks the observed result; the grey dot marks the value implied by the Freedom rank. Values stay on each source’s native scale.

Eval / sub-evalRankValueDistributionSource
COMPL-AI LLM RuLES Multi-Turn Rule Followingscore#3 / 14↓0.2699Source ↗official
DecodingTrusttoxicity#7 / 8↓80Source ↗official
Do-Not-Answerhuman_harmlessness_rate#6 / 6↓99.68Source ↗official
Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate#178 / 270↓89.83Source ↗official
Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate#175 / 270↓85.56Source ↗official
Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate#245 / 268↓99.5Source ↗official
HarmBenchdr#28 / 28↑0.8Source ↗official
JailBenchjailbreak_success_rate#12 / 14↑48.68Source ↗official
MedSafetyBenchmedical_safety_score#30 / 30↓99.81Source ↗official
OR-Benchover_refusal_rate#18 / 25↓87.5Source ↗official
OR-Benchtoxic_acceptance_rate#20 / 25↑0.4Source ↗official
S-Evalbase_en_overall#20 / 22↓78.2Source ↗official
SALAD-Benchattack_enhanced_human_autonomy_integrity#20 / 33↓20.47Source ↗official
SALAD-Benchattack_enhanced_information_safety_harms#19 / 33↓17.59Source ↗official
SALAD-Benchattack_enhanced_malicious_use#20 / 33↓16.31Source ↗official
SALAD-Benchattack_enhanced_misinformation_harms#21 / 33↓18.75Source ↗official
SALAD-Benchattack_enhanced_representation_toxicity#20 / 33↓18.21Source ↗official
SALAD-Benchattack_enhanced_socioeconomic_harms#25 / 33↓23.81Source ↗official
SALAD-Benchbase_human_autonomy_integrity#29 / 33↓98.37Source ↗official
SALAD-Benchbase_information_safety_harms#31 / 33↓99.26Source ↗official
SALAD-Benchbase_malicious_use#30 / 33↓98.54Source ↗official
SALAD-Benchbase_misinformation_harms#25 / 33↓96.26Source ↗official
SALAD-Benchbase_representation_toxicity#25 / 33↓93.06Source ↗official
SALAD-Benchbase_socioeconomic_harms#28 / 33↓94.12Source ↗official
SORRY-Benchavg#38 / 51↑0.14Source ↗official