← Models

Model profile

Command R

Coheredeveloper
2024-03-11release date
#248 / 346Safety rank
#31 / 662Freedom rank

Evidence summary

Safety. Command R has an estimated Safety rank of #248; its 90% source-sensitivity interval is #67–#333. Its behavior-only rank is #249; company governance moves the combined estimate to #248. Published Safety evidence spans 8 eval lineages and 4 of 7 components. Its strongest relative result is AgentDojo (targeted_attack_success_rate, #1 of 15); its weakest is AIRBench 2024 Safety Scenarios (safety_scenarios, #78 of 80).

Freedom. Command R has an estimated Freedom rank of #31; its 90% source-sensitivity interval is #16–#217. Published Freedom evidence spans 6 eval lineages and 1 of 1 components. Its strongest relative result is AIRBench 2024 Safety Scenarios (safety_scenarios, #3 of 80); its weakest is HELM Safety (xstest, #57 of 80).

Compare this model

Only models sharing at least one published sub-eval are listed.

Official and reference links

Safety evals

Rank and direction are specific to the Safety portfolio. Black marks the observed result; the grey dot marks the value implied by the Safety rank. Values stay on each source’s native scale.

Eval / sub-evalRankValueDistributionSource
AgentDojotargeted_attack_success_rate#1 / 15↓0.0334Source ↗official
AgentDojoutility_under_attack#13 / 15↑0.3084Source ↗official
AIRBench 2024 Safety Scenariossafety_scenarios#78 / 80↑0.318Source ↗official
Cisco AI Defense Rolling Single-Turn Leaderboardsingle_turn_attack_success_rate#78 / 104↓62.91Source ↗official
HELM Safetyanthropic_red_team#73 / 80↑0.937Source ↗official
HELM Safetybbq#73 / 80↑0.724Source ↗official
HELM Safetyharmbench#70 / 80↑0.504Source ↗official
HELM Safetysimple_safety_tests#69 / 80↑0.943Source ↗official
HELM Safetyxstest#57 / 80↑0.939Source ↗official

Freedom evals

Rank and direction are specific to the Freedom portfolio. Black marks the observed result; the grey dot marks the value implied by the Freedom rank. Values stay on each source’s native scale.

Eval / sub-evalRankValueDistributionSource
AIRBench 2024 Safety Scenariossafety_scenarios#3 / 80↓0.318Source ↗official
Cisco AI Defense Rolling Single-Turn Leaderboardsingle_turn_attack_success_rate#27 / 104↑62.91Source ↗official
HELM Safetyanthropic_red_team#8 / 80↓0.937Source ↗official
HELM Safetyharmbench#11 / 80↓0.504Source ↗official
HELM Safetysimple_safety_tests#12 / 80↓0.943Source ↗official
HELM Safetyxstest#57 / 80↑0.939Source ↗official