Model profile
GPT 4O
OpenAIdeveloper
2024-05-13release date
#128 / 267overall rank
56eval lineages
Evidence summary
GPT 4O has an estimated overall rank of #128; its 90% source-sensitivity interval is #84–#166. Its behavior-only rank is #138; company governance moves the combined estimate to #128. Published evidence spans 56 evals and 7 of 7 behavior components. Its strongest relative result is OR-Bench (over_refusal_rate, #1 of 25); its weakest is Adversarial Robustness (score, #8 of 8).
Compare this model
Only models sharing at least one published sub-eval are listed.
Official and reference links
- Artificial Analysis ↗gpt-4o
- OpenRouter ↗openai/gpt-4o
- System card ↗Family-level model document · OpenAI · first party
- Release source ↗direct research preview identity date
Published eval results
Rank is within that sub-eval. Black marks the observed result; the grey dot marks the value implied by the global rank. Values stay on each source’s native scale.
| Eval / sub-eval | Rank | Value | Distribution | Better | Source |
|---|---|---|---|---|---|
| AA-Omnisciencehallucination_rate | #29 / 311 | 0.3789 | ↓ lower | Source ↗official | |
| AbstentionBenchanswer_unknown_f1 | #4 / 20 | 0.9017 | ↑ higher | Source ↗official | |
| AbstentionBenchfalse_premise_f1 | #2 / 20 | 0.7607 | ↑ higher | Source ↗official | |
| AbstentionBenchstale_f1 | #5 / 20 | 0.6765 | ↑ higher | Source ↗official | |
| AbstentionBenchsubjective_f1 | #7 / 20 | 0.7572 | ↑ higher | Source ↗official | |
| AbstentionBenchunderspecified_context_f1 | #2 / 20 | 0.7169 | ↑ higher | Source ↗official | |
| AbstentionBenchunderspecified_intent_f1 | #1 / 20 | 0.8152 | ↑ higher | Source ↗official | |
| Adversarial Robustnessscore | #8 / 8 | 67 | ↓ lower | Source ↗official | |
| Agent-SafetyBenchcompromise_availability | #5 / 16 | 35.2 | ↑ higher | Source ↗official | |
| Agent-SafetyBenchharmful_vulnerable_code | #8 / 16 | 35.6 | ↑ higher | Source ↗official | |
| Agent-SafetyBenchleak_sensitive_information | #4 / 16 | 44.4 | ↑ higher | Source ↗official | |
| Agent-SafetyBenchphysical_harm | #3 / 16 | 53.2 | ↑ higher | Source ↗official | |
| Agent-SafetyBenchproduce_unsafe_information | #7 / 16 | 95.6 | ↑ higher | Source ↗official | |
| Agent-SafetyBenchproperty_loss | #3 / 16 | 48.4 | ↑ higher | Source ↗official | |
| Agent-SafetyBenchspread_unsafe_information | #7 / 16 | 12.4 | ↑ higher | Source ↗official | |
| Agent-SafetyBenchviolate_law_ethics | #7 / 16 | 28.8 | ↑ higher | Source ↗official | |
| AgentAbstainabstain | #15 / 17 | 44.2 | ↑ higher | Source ↗official | |
| AgentAbstaincar | #17 / 17 | 40.9 | ↑ higher | Source ↗official | |
| AgentAbstainpaired | #17 / 17 | 33 | ↑ higher | Source ↗official | |
| AgentDojotargeted_attack_success_rate | #15 / 15 | 0.4769 | ↓ lower | Source ↗official | |
| AgentDojoutility_under_attack | #4 / 15 | 0.5008 | ↑ higher | Source ↗official | |
| AgentHarmharm_score | #8 / 12 | 48.4 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatchild_sexual_exploitation | #9 / 32 | 12 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatdefamation | #16 / 32 | 15.86 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chathate | #17 / 32 | 9 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatindiscriminate_weapons_cbrne | #12 / 32 | 19.2 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatintellectual_property | #15 / 32 | 10.5 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatnon_violent_crimes | #14 / 32 | 15.6 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatprivacy | #12 / 32 | 10.2 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatsex_related_crimes | #16 / 32 | 14.3 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatsexual_content | #21 / 32 | 33.6 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatspecialized_advice | #18 / 32 | 17.62 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatsuicide_self_harm | #15 / 32 | 12.1 | ↓ lower | Source ↗official | |
| AILuminate General Purpose AI Chatviolent_crimes | #12 / 32 | 14.9 | ↓ lower | Source ↗official | |
| AIRBench 2024 Safety Scenariossafety_scenarios | #55 / 80 | 0.5755 | ↑ higher | Source ↗official | |
| Alignment Leaderboardcorrigibility | #12 / 24 | 4.243 | ↑ higher | Source ↗official | |
| Alignment Leaderboardhonesty | #13 / 24 | 3.604 | ↑ higher | Source ↗official | |
| Alignment Leaderboardnon_manipulation | #15 / 24 | 3.276 | ↑ higher | Source ↗official | |
| Alignment Leaderboardrobustness | #2 / 24 | 4.56 | ↑ higher | Source ↗official | |
| Alignment Leaderboardsafety | #14 / 24 | 3.808 | ↑ higher | Source ↗official | |
| Alignment Leaderboardscheming | #15 / 24 | 3.584 | ↑ higher | Source ↗official | |
| AnimalHarmBenchscore | #8 / 10 | 0.011 | ↑ higher | Source ↗official | |
| Anthropic Agentic Misalignment — blackmailmisaligned_action_rate_pct | #4 / 16 | 15 | ↓ lower | Source ↗official | |
| Anthropic Agentic Misalignment — corporate espionagemisaligned_action_rate_pct | #2 / 16 | 6 | ↓ lower | Source ↗official | |
| BlueBench AttaQ-100attaq_harmlessness_reward_pct | #4 / 18 | 87.84 | ↑ higher | Source ↗official | |
| BullshitBench v2clear_pushback_rate | #81 / 105 | 0.12 | ↑ higher | Source ↗official | |
| CAIS Risk Indexagent_red_teaming | #34 / 43 | 90.7 | ↓ lower | Source ↗official | |
| CAIS Risk Indexbioweapons_assistance | #40 / 48 | 92.7 | ↓ lower | Source ↗official | |
| CAIS Risk Indexhle_overconfidence | #49 / 49 | 89 | ↓ lower | Source ↗official | |
| CAIS Risk Indexmachiavelli | #42 / 45 | 99.6 | ↓ lower | Source ↗official | |
| CAIS Risk Indexmask | #51 / 51 | 78 | ↓ lower | Source ↗official | |
| CAIS Risk Indexpolitical_manipulation | #29 / 32 | 59.6 | ↓ lower | Source ↗official | |
| CAIS Risk Indextextquests_harm | #5 / 48 | 11.2 | ↓ lower | Source ↗official | |
| CASE-Benchagreement_accuracy | #7 / 7 | 78.56 | ↑ higher | Source ↗official | |
| Cisco AI Defense Rolling Single-Turn Leaderboardsingle_turn_attack_success_rate | #60 / 105 | 42.02 | ↓ lower | Source ↗official | |
| Confabulationsconfabulation_rate | #26 / 52 | 24.26 | ↓ lower | Source ↗official | |
| CRiskEvaldeception_willingness | #1 / 17 | 10.6 | ↓ lower | Source ↗official | |
| CRiskEvaldesire_for_resource | #1 / 17 | 19.04 | ↓ lower | Source ↗official | |
| CRiskEvalharmful_goal | #2 / 17 | 27.23 | ↓ lower | Source ↗official | |
| CRiskEvalimprovement_intent | #3 / 17 | 43.32 | ↓ lower | Source ↗official | |
| CRiskEvalmalicious_coordination | #4 / 17 | 6.71 | ↓ lower | Source ↗official | |
| CRiskEvalself_preservation | #2 / 17 | 25.92 | ↓ lower | Source ↗official | |
| CRiskEvalsituational_awareness | #2 / 17 | 38.75 | ↓ lower | Source ↗official | |
| Enkrypt AI Safety Leaderboardbias_attack_non_success_rate | #86 / 260 | 20.67 | ↑ higher | Source ↗official | |
| Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate | #33 / 260 | 93.83 | ↑ higher | Source ↗official | |
| Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate | #140 / 260 | 67.78 | ↑ higher | Source ↗official | |
| Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate | #59 / 258 | 98.41 | ↑ higher | Source ↗official | |
| FORTRESSaverage_risk_score | #33 / 49 | 47.18 | ↓ lower | Source ↗official | |
| FORTRESSover_refusal_score | #9 / 46 | 1.68 | ↓ lower | Source ↗official | |
| HELM Safetyanthropic_red_team | #32 / 80 | 0.991 | ↑ higher | Source ↗official | |
| HELM Safetybbq | #26 / 80 | 0.951 | ↑ higher | Source ↗official | |
| HELM Safetyharmbench | #33 / 80 | 0.829 | ↑ higher | Source ↗official | |
| HELM Safetysimple_safety_tests | #43 / 80 | 0.985 | ↑ higher | Source ↗official | |
| HELM Safetyxstest | #19 / 80 | 0.973 | ↑ higher | Source ↗official | |
| HUMAINE Trust, Ethics and Safetytrust_ethics_safety_score | #53 / 54 | 22.28 | ↑ higher | Source ↗official | |
| Large-scale Moral Machine experiment on LLMshuman_choice_distance | #17 / 39 | 0.9418 | ↓ lower | Source ↗official | |
| LLM Ethics Benchmarkscore | #2 / 5 | 90 | ↑ higher | Source ↗official | |
| MACHIAVELLIdeception_relative_random_pct | #44 / 50 | 99.6 | ↓ lower | Source ↗official | |
| Manager Coercion Benchcoercion_ladder_depth | #12 / 31 | 7.967 | ↓ lower | Source ↗self run | |
| MASKlying_probability_pct | #29 / 53 | 42.22 | ↓ lower | Source ↗official | |
| OpenAgentSafetyllm_judge_safety_vulnerable | #6 / 7 | 65.8 | ↓ lower | Source ↗official | |
| OpenAgentSafetyrule_based_safety_vulnerable | #3 / 7 | 32.64 | ↓ lower | Source ↗official | |
| OpenAgentSafetysuccessful_completion | #5 / 7 | 21.77 | ↑ higher | Source ↗official | |
| OpenAI GPT-4o System Cardspeaker_privacy_refusal_accuracy | #1 / 2 | 0.98 | ↑ higher | Source ↗official | |
| OpenAI GPT-5 System Cardharmful_request_safety | #2 / 2 | 0.633 | ↑ higher | Source ↗official | |
| OpenAI o3-mini System Cardharmful_request_safety | #2 / 2 | 0.86 | ↑ higher | Source ↗official | |
| OpenAI o3-mini System Cardjailbreak_resistance | #2 / 2 | 0.51 | ↑ higher | Source ↗official | |
| OR-Benchover_refusal_rate | #1 / 25 | 6.8 | ↓ lower | Source ↗official | |
| OR-Benchtoxic_acceptance_rate | #19 / 25 | 15.1 | ↓ lower | Source ↗official | |
| PandaBench JBB direct-request panelsafety_rate | #13 / 46 | 0.995 | ↑ higher | Source ↗official | |
| PHAREbias_resistance_diagnostic | #24 / 66 | 0.5092 | ↑ higher | Source ↗official | |
| PHAREhallucination_resistance_diagnostic | #24 / 70 | 0.7852 | ↑ higher | Source ↗official | |
| PHAREharm_resistance_diagnostic | #38 / 70 | 0.9266 | ↑ higher | Source ↗official | |
| PHAREjailbreak_resistance_diagnostic | #25 / 67 | 0.6048 | ↑ higher | Source ↗official | |
| PropensityBenchscore | #7 / 14 | 46.1 | ↓ lower | Source ↗official | |
| S-Evalbase_en_overall | #16 / 22 | 52 | ↑ higher | Source ↗official | |
| SafeArenanormalized_safety_score | #4 / 5 | 31.7 | ↑ higher | Source ↗official | |
| SafeDialBenchaggression | #4 / 18 | 7.207 | ↑ higher | Source ↗official | |
| SafeDialBenchethics | #12 / 18 | 7.487 | ↑ higher | Source ↗official | |
| SafeDialBenchfairness | #2 / 18 | 7.68 | ↑ higher | Source ↗official | |
| SafeDialBenchlegality | #15 / 18 | 7.21 | ↑ higher | Source ↗official | |
| SafeDialBenchmorality | #10 / 18 | 7.237 | ↑ higher | Source ↗official | |
| SafeDialBenchprivacy | #14 / 18 | 7.14 | ↑ higher | Source ↗official | |
| Shelleducation_jsr | #12 / 14 | 0.804 | ↓ lower | Source ↗official | |
| Shellfinance_jsr | #12 / 14 | 0.826 | ↓ lower | Source ↗official | |
| Shellmanagement_jsr | #12 / 14 | 0.872 | ↓ lower | Source ↗official | |
| SM-Benchadversarial | #59 / 73 | 77.07 | ↑ higher | Source ↗official | |
| SM-Benchambiguous_interpretation | #72 / 73 | 58.93 | ↑ higher | Source ↗official | |
| SM-Benchanti_hallucination | #54 / 73 | 86.39 | ↑ higher | Source ↗official | |
| SM-Bencheq_boundaries | #67 / 73 | 51.12 | ↑ higher | Source ↗official | |
| SM-Benchoverfit | #18 / 73 | 83.06 | ↑ higher | Source ↗official | |
| Social Welfare Function Benchmarkfairness | #9 / 19 | 0.491 | ↑ higher | Source ↗official | |
| SORRY-Benchavg | #27 / 51 | 0.3 | ↓ lower | Source ↗official | |
| SOSBenchbiology_pvr | #8 / 23 | 0.306 | ↓ lower | Source ↗official | |
| SOSBenchchemistry_pvr | #8 / 23 | 0.254 | ↓ lower | Source ↗official | |
| SOSBenchmedicine_pvr | #9 / 23 | 0.476 | ↓ lower | Source ↗official | |
| SOSBenchpharmacology_pvr | #9 / 23 | 0.676 | ↓ lower | Source ↗official | |
| SOSBenchphysics_pvr | #6 / 23 | 0.194 | ↓ lower | Source ↗official | |
| SOSBenchpsychology_pvr | #10 / 23 | 0.396 | ↓ lower | Source ↗official | |
| SpeciesismBenchexplicit_speciesism_scale | #3 / 7 | 2.383 | ↓ lower | Source ↗official | |
| SpeciesismBenchmorally_wrong_rate | #6 / 8 | 35.96 | ↑ higher | Source ↗official | |
| SpeciesismBenchspeciesism_recognition_rate | #1 / 8 | 98.37 | ↑ higher | Source ↗official | |
| SYCON Benchfalse_presupposition_tof | #3 / 11 | 2.92 | ↑ higher | Source ↗official | |
| SYCON Benchunethical_queries_tof | #8 / 11 | 1.23 | ↑ higher | Source ↗official | |
| TukaBenchafri_jbb_cultural_asr | #5 / 6 | 27.5 | ↓ lower | Source ↗official | |
| TukaBenchafri_jbb_harm_asr | #3 / 6 | 13.5 | ↓ lower | Source ↗official | |
| TukaBenchafrijail_mono_asr | #5 / 6 | 25.7 | ↓ lower | Source ↗official | |
| UAVBench safety-critical decision recognitionethical_safety_critical_accuracy | #8 / 27 | 0.73 | ↑ higher | Source ↗official | |
| Vigil Mental Health Safetyoverall_score | #21 / 23 | 27 | ↑ higher | Source ↗official |
Values evaluations
Descriptive values and political-framing results are separate from safety/ethics ranks. Each strip shows the evaluation’s observed model range; its endpoint labels state what lower and higher values mean.
UGI Political Values
ValueCompass
| Dimension | Value | Distribution |
|---|---|---|
| Universalism | 67.9 | |
| Self-direction | 51.1 | |
| Care / Harm | 37.1 | |
| Fairness / Cheating | 35.5 | |
| Ethical | 90.8 |
Taiwan Sovereignty Benchmark Pro
| Dimension | Value | Distribution |
|---|---|---|
| Pro-Taiwan rubric compatibility | 90 | |
| Warning-phrase rate | 0 | |
| Soft-censorship rate | 0 | |
| API-error rate | 0 |
CCP-aligned censorship behavior
| Dimension | Value | Distribution |
|---|---|---|
| Political-question refusal rate (ZH/EN mean) | 0.0655 | |
| Completely inaccurate rate | 7.82 |