How Humanoid Arena scores robots
Humanoid Arena scores humanoid robots by public evidence of human-like capability. It is a structured evidence system, not a physical benchmark or certification.
Public evidence of humanoid capability
The leaderboard compares public specs, demos, deployments, customer claims, company materials, and structured LLM-assisted pairwise judgments. The goal is to make it easier to understand which humanoids currently appear most capable, what evidence supports that view, and where the evidence is still weak.
Humanoids with enough public information to compare meaningfully.
Structured claims with source quality, caveats, and missing-data notes.
Elo-style scores derived from evidence-weighted pairwise outcomes.
What "human-like capability" means
Humanoid Arena defines human-like capability as a robot's demonstrated ability to perceive its surroundings, move through human-designed spaces, manipulate objects, follow instructions, communicate with people, complete useful tasks, and operate reliably outside a one-off demo.
We do not define capability as looking human, having a human face, or appearing impressive in a promotional video. A robot receives stronger scores when public evidence supports useful physical capability in real-world or realistic environments.
| Dimension | Weight | What it measures |
|---|---|---|
| Manipulation / dexterity | 25% | Hands, grasping, object handling, tool/task usefulness |
| Autonomy / task execution | 20% | Can it complete tasks without constant teleoperation? |
| Mobility / whole-body control | 15% | Walking, balance, human-space movement |
| Perception / scene understanding | 15% | Vision, spatial awareness, object recognition |
| Human interaction | 10% | Speech, instruction following, social usability |
| Endurance / reliability | 10% | Runtime, operating hours, repeatability |
| Human-environment fit | 5% | Size, safety, form factor, ability to work around people |
What the Arena Rating means
Each robot receives a Humanoid Arena Rating. The rating is an Elo-style number derived from many pairwise comparisons. A higher rating means the robot tends to win more evidence-based head-to-head judgments under the rubric.
| Rating range | Rough meaning |
|---|---|
| 500-700 | Early prototype, limited public capability evidence |
| 700-900 | Demonstrated humanoid platform, but narrow or weak evidence |
| 900-1100 | Credible current humanoid with meaningful demos or specs |
| 1100-1300 | Strong humanoid with good evidence across several dimensions |
| 1300+ | Leading humanoid with unusually strong public evidence |
| 1500+ | Reserved for broad, repeatable, well-documented real-world capability |
The exact number is less important than confidence range, matchup history, category strengths, caveats, and the public evidence trail.
Structured evidence summaries
Each robot is represented by an internal structured evidence summary: robot generation, status, company information, technical specs, public demos, deployment claims, customer or partner evidence, availability, strengths, caveats, missing evidence, and source quality.
LLM judges do not simply vote based on robot names. They compare structured summaries. The product is designed to reward demonstrated or credibly documented capability, not brand awareness or polished marketing.
Evidence tiers
| Tier | Evidence type | How it is treated |
|---|---|---|
| Tier 5 | Independent benchmark or third-party measured test | Strongest evidence |
| Tier 4 | Customer-confirmed deployment with task, hours, output, or operating data | Very strong evidence |
| Tier 3 | Company-published deployment data with measurable numbers | Strong but company-sourced |
| Tier 2 | Clear public demo video or public demo | Useful but limited evidence |
| Tier 1 | Official spec sheet, product page, roadmap, or press release | Basic supporting evidence |
| Tier 0 | Unsourced claim, rumor, vague social post, or hype | Weak or excluded |
How head-to-head voting works
Humanoid Arena generates randomized head-to-head comparisons between robots. A comparison asks: given the evidence provided, which robot shows stronger human-like capability for this scenario?
| Result | Meaning |
|---|---|
| Robot A wins | A has stronger evidence for the scenario |
| Robot B wins | B has stronger evidence for the scenario |
| Draw | Evidence suggests neither robot clearly wins |
| Insufficient evidence | The comparison should not affect ratings |
Top weights: Manipulation 25% · Autonomy 20% · Mobility 15%
Top weights: Manipulation 25% · Reliability 25% · Autonomy 20%
Top weights: Autonomy 24% · Manipulation 22% · Perception 18%
Top weights: Manipulation 30% · Autonomy 25% · Perception 20%
Top weights: Human interaction 30% · Autonomy 25% · Perception 20%
Top weights: Manipulation 50% · Perception 20% · Autonomy 15%
Blind and named comparisons
Some comparisons run in blind mode, where the judge sees the evidence but not robot or company names. This reduces brand bias. Other comparisons run in named mode because company context can matter for deployments, availability, or generation history. The system tracks both.
How LLM judges are instructed
Judges are instructed to use only the supplied structured summary, avoid outside knowledge, avoid brand familiarity, separate demonstrated capability from future promises, reduce confidence when evidence is missing, and allow draws or insufficient-evidence outcomes.
How ratings are calculated
The system starts with pairwise outcomes. Across many comparisons, those outcomes are aggregated into a Bradley-Terry/Elo-style rating model. If a robot consistently beats strong robots, its rating rises more than if it only beats weak robots.
The public rating is shown as an Elo-style score because it is easy to understand. Internally, the intended model estimates relative strength from pairwise comparisons and can produce confidence intervals through bootstrapping.
Evidence-weighted votes
Not every vote has the same strength. Votes based on stronger evidence move ratings more. Customer-confirmed deployment matters more than a short edited demo, and specs alone do not prove reliable task execution.
| Evidence basis | Rating impact |
|---|---|
| Customer-confirmed deployment | High |
| Company-published deployment numbers | Medium-high |
| Clear public demo | Medium |
| Spec sheet only | Low |
| Rumor or vague claim | Very low or excluded |
Confidence and evidence strength
Confidence depends on comparison volume, evidence strength, consistency of outcomes, missing data, agreement between blind and named comparisons, recency, and whether customer or third-party validation exists.
| Evidence Strength | Meaning |
|---|---|
| A | Strong public evidence, usually including deployment or high-quality third-party support |
| B | Good evidence, but still mostly company-controlled or demo-based |
| C | Some meaningful evidence, but major gaps remain |
| D | Sparse, unclear, or mostly speculative evidence |
| Unknown | Not enough information |
Evidence Strength is separate from Arena Rating. A robot can look promising but be weakly proven. Another robot can be less flashy but better supported by deployment data.
Weekly updates
Humanoid Arena is designed to update as new launches, specs, demos, customer deployments, pricing, corrections, retirements, and head-to-head votes arrive. Large rating jumps should normally require strong evidence such as customer-confirmed deployment, measurable operating data, or a major new capability demonstration.
Verified deployment, repeated task performance, useful autonomous manipulation, realistic mobility, operating-hour evidence, purchasability, or strong blind-mode results.
Promotional evidence, edited demos, unclear teleoperation, undisclosed autonomy, missing specs, no deployment evidence, supersession, or claims that are hard to verify.
What we do not claim
The leaderboard is not a laboratory benchmark.
Robots have not been physically tested by Humanoid Arena.
Ratings are not perfect objective truth.
A higher current rating does not mean a robot is better for every use case.
Demos do not prove commercial readiness.
Company claims are not independently verified unless stated.
LLM judges are not assumed to be bias-free.
The current ordering is best understood as a structured, transparent view of public evidence. It is not a replacement for real-world testing.
FAQ
Is this a real-world benchmark?
No. Humanoid Arena does not physically test robots. It scores robots using public evidence, structured summaries, and LLM-assisted pairwise comparisons.
Why use LLM judges?
LLMs are useful for comparing structured evidence across many robots and scenarios. Their votes are constrained by rubrics, evidence tiers, validation checks, and confidence scoring.
Can a marketing video affect the rating?
Yes, but only modestly. Demo videos are weaker than customer-confirmed deployments, operating-hour data, or independent benchmarks.
What happens when evidence is missing?
Missing evidence reduces confidence. It does not automatically prove the robot is bad, but unproven capabilities receive limited credit.
Why not just sort by specs?
Specs are useful, but they do not prove that a robot can reliably complete tasks.
Can ratings change?
Yes. Ratings and ordering update as new evidence appears and as the arena runs new comparisons.
Based on public evidence, which humanoids currently appear most capable, and why?
Humanoid Arena combines structured robot summaries, evidence quality tiers, randomized pairwise comparisons, blind and named LLM judging, scenario-specific scoring, evidence-weighted votes, Bradley-Terry/Elo-style ratings, and confidence indicators to make public evidence easier to inspect.