Model profile
Deepseek V3
DeepSeekdeveloper
2024-12-26release date
#120 / 267overall rank
30eval lineages
Evidence summary
Deepseek V3 has an estimated overall rank of #120; its 90% source-sensitivity interval is #57–#178. Its behavior-only rank is #105; company governance moves the combined estimate to #120. Published evidence spans 30 evals and 7 of 7 behavior components. Its strongest relative result is Contextual MoralChoice (human_agreement, #2 of 22); its weakest is SOSBench (medicine_pvr, #23 of 23).
Compare this model
Only models sharing at least one published sub-eval are listed.
Official and reference links
- Artificial Analysis ↗deepseek-v3
- Hugging Face ↗deepseek-ai/DeepSeek-V3
- OpenRouter ↗deepseek/deepseek-chat-v3-0324
- Official model page ↗Exact model document · Reviewed official Hugging Face owner · official repository
- Release source ↗direct research preview identity date
Published eval results
Rank is within that sub-eval. Black marks the observed result; the grey dot marks the value implied by the global rank. Values stay on each source’s native scale.
| Eval / sub-eval | Rank | Value | Distribution | Better | Source |
|---|---|---|---|---|---|
| AA-Omnisciencehallucination_rate | #127 / 311 | 0.7675 | ↓ lower | Source ↗official | |
| AIRBench 2024 Safety Scenariossafety_scenarios | #73 / 80 | 0.408 | ↑ higher | Source ↗official | |
| AnimalHarmBenchscore | #5 / 10 | 0.04 | ↑ higher | Source ↗official | |
| BullshitBench v2clear_pushback_rate | #100 / 105 | 0.04 | ↑ higher | Source ↗official | |
| ChiSafetyBenchharmful_response_rate | #5 / 14 | 0.43 | ↓ lower | Source ↗official | |
| ChiSafetyBenchmcq_score | #6 / 12 | 84.17 | ↑ higher | Source ↗official | |
| Confabulationsconfabulation_rate | #44 / 52 | 39.11 | ↓ lower | Source ↗official | |
| Contextual MoralChoicehuman_agreement | #2 / 22 | 0.6 | ↑ higher | Source ↗official | |
| Emergent Collusionhigh_illegality_game_rate | #3 / 13 | 0.26 | ↓ lower | Source ↗official | |
| Enkrypt AI Safety Leaderboardbias_attack_non_success_rate | #172 / 260 | 13.18 | ↑ higher | Source ↗official | |
| Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate | #148 / 260 | 87.33 | ↑ higher | Source ↗official | |
| Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate | #190 / 260 | 52.22 | ↑ higher | Source ↗official | |
| Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate | #218 / 258 | 89.64 | ↑ higher | Source ↗official | |
| HELM Safetyanthropic_red_team | #61 / 80 | 0.971 | ↑ higher | Source ↗official | |
| HELM Safetybbq | #13 / 80 | 0.967 | ↑ higher | Source ↗official | |
| HELM Safetyharmbench | #72 / 80 | 0.497 | ↑ higher | Source ↗official | |
| HELM Safetysimple_safety_tests | #67 / 80 | 0.953 | ↑ higher | Source ↗official | |
| HELM Safetyxstest | #22 / 80 | 0.971 | ↑ higher | Source ↗official | |
| HUMAINE Trust, Ethics and Safetytrust_ethics_safety_score | #22 / 54 | 27.39 | ↑ higher | Source ↗official | |
| JailBenchjailbreak_success_rate | #10 / 14 | 70.67 | ↓ lower | Source ↗official | |
| LiveSecBenchethics | #38 / 43 | 20.88 | ↑ higher | Source ↗official | |
| LiveSecBenchfactuality | #34 / 43 | 29.99 | ↑ higher | Source ↗official | |
| LiveSecBenchlegality | #42 / 43 | 7.76 | ↑ higher | Source ↗official | |
| LiveSecBenchprivacy | #42 / 43 | 6.72 | ↑ higher | Source ↗official | |
| LiveSecBenchpsychological_health | #36 / 43 | 25.03 | ↑ higher | Source ↗official | |
| LLM Ethics Benchmarkscore | #3 / 5 | 86.1 | ↑ higher | Source ↗official | |
| MASKlying_probability_pct | #49 / 53 | 54.48 | ↓ lower | Source ↗official | |
| OpenAgentSafetyllm_judge_safety_vulnerable | #4 / 7 | 62.23 | ↓ lower | Source ↗official | |
| OpenAgentSafetyrule_based_safety_vulnerable | #2 / 7 | 32.44 | ↓ lower | Source ↗official | |
| OpenAgentSafetysuccessful_completion | #4 / 7 | 22.12 | ↑ higher | Source ↗official | |
| PandaBench JBB direct-request panelsafety_rate | #24 / 46 | 0.985 | ↑ higher | Source ↗official | |
| PHAREbias_resistance_diagnostic | #13 / 66 | 0.5751 | ↑ higher | Source ↗official | |
| PHAREhallucination_resistance_diagnostic | #57 / 70 | 0.6675 | ↑ higher | Source ↗official | |
| PHAREharm_resistance_diagnostic | #48 / 70 | 0.909 | ↑ higher | Source ↗official | |
| PHAREjailbreak_resistance_diagnostic | #62 / 67 | 0.3107 | ↑ higher | Source ↗official | |
| Social Welfare Function Benchmarkfairness | #3 / 19 | 0.594 | ↑ higher | Source ↗official | |
| SOSBenchbiology_pvr | #22 / 23 | 0.856 | ↓ lower | Source ↗official | |
| SOSBenchchemistry_pvr | #17 / 23 | 0.6 | ↓ lower | Source ↗official | |
| SOSBenchmedicine_pvr | #23 / 23 | 0.872 | ↓ lower | Source ↗official | |
| SOSBenchpharmacology_pvr | #15 / 23 | 0.916 | ↓ lower | Source ↗official | |
| SOSBenchphysics_pvr | #17 / 23 | 0.722 | ↓ lower | Source ↗official | |
| SOSBenchpsychology_pvr | #21 / 23 | 0.82 | ↓ lower | Source ↗official | |
| SpeciesismBenchmorally_wrong_rate | #7 / 8 | 32.7 | ↑ higher | Source ↗official | |
| SpeciesismBenchspeciesism_recognition_rate | #5 / 8 | 87.71 | ↑ higher | Source ↗official | |
| SpeciEvalbelief_animal_sentience | #87 / 102 | 6.42 | ↑ higher | Source ↗official | |
| SpeciEvalland_animal_4ns | #85 / 102 | 4.86 | ↓ lower | Source ↗official | |
| SpeciEvalsea_animal_4ns | #89 / 102 | 5.1 | ↓ lower | Source ↗official | |
| SpeciEvalspeciesism | #82 / 102 | 2.54 | ↓ lower | Source ↗official | |
| SYCON Benchfalse_presupposition_tof | #5 / 11 | 2.88 | ↑ higher | Source ↗official | |
| SYCON Benchunethical_queries_tof | #5 / 11 | 1.99 | ↑ higher | Source ↗official | |
| TACbase_welfare_rate | #7 / 68 | 42.3 | ↑ higher | Source ↗self run | |
| UAVBench safety-critical decision recognitionethical_safety_critical_accuracy | #4 / 27 | 0.755 | ↑ higher | Source ↗official | |
| VETO Misfired Alignmentmisfired_alignment_rate_pct | #3 / 25 | 5.2 | ↓ lower | Source ↗official |
Values evaluations
Descriptive values and political-framing results are separate from safety/ethics ranks. Each strip shows the evaluation’s observed model range; its endpoint labels state what lower and higher values mean.
UGI Political Values
ValueCompass
| Dimension | Value | Distribution |
|---|---|---|
| Universalism | 70.4 | |
| Self-direction | 47.7 | |
| Care / Harm | 34.5 | |
| Fairness / Cheating | 32.2 | |
| Ethical | 91.7 |