正文 · AI 翻译
Yunzhong reporting from Aofei Temple
QbitAI | WeChat public account QbitAI
Intuitive judgments that humans make in daily life often become difficult problems for robots.
An object appearing higher in a video frame does not mean it is higher in real three-dimensional space; furniture may look different from another camera angle, yet still be the same object; an object shifting within the frame may mean the object itself moved, or merely that the camera turned…
Humans can distinguish these subtle changes in space and state at a glance, but they are difficult problems for a robot’s “embodied brain.”
Confusing an object’s identity when the viewpoint changes, misjudging camera movement, or misestimating task progress can all directly cause a robot to fail.
For robots to complete real-world tasks reliably, a model must do more than “see objects”: it must understand three-dimensional spatial relationships, detect changes in object states, and determine what stage a task has reached.
Recently, the Lion Rock AI Lab under the China Merchants Advanced Technology Research Institute released the embodied reasoning vision-language model “Yuwu™ Lion-ER-35B-A3B.” It uses a mixture-of-experts (MoE) architecture with approximately 35B total parameters, activating only about 3B parameters per token.
Without significantly increasing inference costs, the model improved on every metric across 10 mainstream embodied-AI benchmarks. Several core capabilities surpassed those of mainstream models with more parameters, charting a technical path distinct from simply amassing training data and helping robots truly understand the three-dimensional physical world.
How was this model developed?
Over the past few years, projects such as Open X-Embodiment and DROID have continued to expand the boundaries of real-world robot data. But as datasets have grown, one issue has become increasingly clear:
More data does not necessarily mean greater capability.
Existing research shows that, even within existing robot datasets, merely adjusting the training weights assigned to different data domains can noticeably affect performance on downstream tasks.
The main strength of “Yuwu™ Lion-ER-35B-A3B” is not simply a larger parameter count. The team built a capability-driven embodied-data engine, moving away from indiscriminately collecting vast amounts of data. Instead, it designs data around the capabilities the model needs to master, then uses evaluations to identify weaknesses and iteratively improve the data in a closed loop.
Step 1: Define what the model should learn
The China Merchants Lion Rock team mapped out a fine-grained “capability blueprint”:
They divided embodied reasoning into 8 top-level domains—entities, space, time, physics, affordances, tasks, social reasoning, and evidence—and refined these into 39 secondary capabilities. They also labeled reasoning methods, input formats, and answer formats as separate dimensions to avoid overlap caused by mixing classification dimensions.
The blueprint defines not only which capabilities the model needs to learn, but also the conditions under which those capabilities should be used.
Real tasks often require combined capabilities. For example, determining whether an item has been placed in the right position requires both an assessment of task progress and reasoning about spatial relationships.
The team therefore recorded primary and supporting capabilities separately at the blueprint stage, and listed joint tasks that cannot be broken down on their own.
When sampling training data, the team also considers combinations and held-out cases involving different factors, so that a capability is not always tied to a small set of fixed scenes, objects, or ways of asking questions.
Step 2: Turn abstract requirements into individual training questions
With a capability blueprint in place, how can the team put it into practice to generate training questions?
The team calls this process “capability compilation”:
It breaks down each abstract capability goal into concrete specifications: what scene is needed, how observations should be organized, what question should be asked, and what evidence should determine the answer.
Take cross-view reasoning as an example. First, the team specifies how many views the model will receive, which target object must be linked across multiple images, and the observer’s position and facing direction used to determine orientation.
The source material must also be checked: Is the visual information given to the model sufficient for it to derive a reliable answer? If a key view is missing or the evidence is insufficient, answering the question would require guessing, so it cannot be included as a valid training sample.
These rules for constructing questions are driven by the target capability itself; the team does not need to wait for evaluations to reveal errors before adding data reactively.
The capability blueprint plans the range of capabilities to cover in advance, while evaluations identify gaps and pinpoint areas where the model remains weak.
Step 3: Ensure every training question is supported by evidence
For constructing questions, the team designed 3 approaches that can be combined: converting existing Q&A and visual annotations, finding evidence in observation material, and deriving questions from structured conditions and relationships. Together, these approaches turn abstract capability requirements into specific, answerable training questions.
For verifying supervision, the team uses 3 complementary approaches: data refinement, geometry-based question construction, and targeted synthesis:
Data refinement: Clean and revise existing data, check image order, references in questions, and mappings between options and answers, correct errors where supporting evidence exists, and limit repeated exposure.
Geometry-based question construction: For material with reliable pose, depth, and object correspondences, first calculate a fixed answer and the intermediate relationships needed for reasoning, then have a model generate questions and explanations constrained by those facts.
Targeted synthesis: When existing supervision is insufficient, construct questions and answers around real images or video. The questions and annotations are synthesized, but answers from the generative model are never treated directly as ground truth; they must undergo independent answering or additional review.
The entire process follows one principle: retain difficult questions backed by evidence and exclude questions that only appear difficult because evidence is missing.
A model getting a question wrong is no reason to discard the sample, and multiple models giving the same answer does not, by itself, prove the answer is correct.
For difficult samples on which models repeatedly fail, the team adds human-in-the-loop review. It built a visual verification platform for complex embodied tasks, using multi-view comparisons, temporal playback, and evidence markers to strengthen human review. These tools help reviewers reconstruct key visual clues and the basis for reasoning, so that limits in model capability are not mistaken for data problems.
Step 4: Feed evaluation results directly into data iteration
Once valid training questions are available, training resources must also be allocated appropriately.
The supervised training set for “Yuwu™ Lion-ER-35B-A3B” contains 312380 samples. During training, the budget is allocated according to capability needs, rather than letting the number of questions from each data source determine its weight.
The samples focus on core embodied capabilities such as spatial geometry, cross-view correspondence, action trajectories, and task planning. The team reduces repetitive, similar questions and directs training resources toward weak types of reasoning. Each sample is tracked from candidate selection through review to inclusion in training, making the training recipe traceable and adjustable.
Forward design establishes capability coverage, but has the model actually learned those capabilities, and where do blind spots remain?
That is the role of the Eval-Driven Data Harness.
This data-iteration mechanism identifies capabilities that need strengthening from questions the model gets wrong, then turns them into actionable data requirements for the next round.
In the Harness, evaluation is no longer a report card issued after training; it is the starting point for the next round of data production.
When the model gets a question wrong, the team first checks the annotations, visual evidence, and scoring for problems, then distinguishes occasional mistakes from systematic capability gaps. Once a gap is confirmed, the team searches a source pool spanning 31 data sources and about 4.3 million records for new material with the same reasoning logic but different scenes. After question construction and verification, the material is added to training, and its effect is checked in another evaluation round.
For example, if the model tends to equate an object’s vertical position in a two-dimensional image with its height in three-dimensional space, the team prioritizes scenes with multiple vertically stacked surfaces on which objects can be placed, requiring the model to judge height from spatial structure. It then adds occlusion and perspective changes to left-right and front-back relationships to cover cases that are easier to confuse.
The team also combines segmentation, depth, and geometric constraints to check whether placement areas make sense.
In short, the capability blueprint sets the direction of coverage, while the Eval-Driven Data Harness continually addresses blind spots.
Data is no longer a one-time, static supply; it evolves continuously with the model’s capabilities.
Only about 3B parameters activated; improvements across all 10 benchmarks
With this capability-driven data engine, “Yuwu™ Lion-ER-35B-A3B” activates only about 3B parameters during inference yet improves on every metric across 10 public benchmarks, with some metrics surpassing models with more parameters.
The evaluation uses avg@4: the model answers each question independently 4 times, and the scores are averaged.
Across the 10 public benchmarks in the evaluation, “Yuwu™ Lion-ER-35B-A3B” achieved an equally weighted composite average score of 65.29.
Looking at individual results:
It scored 64.28 on the Where2Place spatial-placement benchmark, up 9.30 points from the base model;
It scored 64.38 on the ERQA embodied-reasoning benchmark, up 6.19 points from the base model;
It reached 89.33 on the SAT spatial-reasoning benchmark;
It reached 68.18 on ERQA’s trajectory-understanding subcategory, up 12.12 points from the base model.
Overall, “Yuwu™ Lion-ER-35B-A3B” scored higher than its base model on all 10 benchmarks.
4 public benchmark questions that show the capability gap directly
The following 4 examples all come from public evaluation benchmarks. For each question, “Yuwu™ Lion-ER-35B-A3B,” the base model Qwen3.6-35B-A3B, and Qwen3.6-27B each answered independently 4 times.
Question 1: Determine the direction—which way is the camera turning?
Two images were captured consecutively by a camera mounted on a robotic arm from its point of view. The bottle shifts upward overall in the frame. The question is which direction the camera is turning.
The base model answered correctly only once, and Qwen3.6-27B got every attempt wrong. By analyzing how the two frames changed, “Yuwu™ Lion-ER-35B-A3B” correctly determined that the camera was turning downward, answering correctly all 4 times.
When an object’s position changes in a robot’s camera view, the robot must first determine the cause: whether the object itself moved or the camera turned.
Question 2: Choose a trajectory—which path can complete the task?
Four colored trajectories are marked in the image. The robotic arm’s goal is to unplug a charger from a power strip and place it on the right side of the image. Which trajectory should it choose?
Both comparison models were misled by the long purple arc extending to the right and chose that trajectory. It appears to extend toward the target placement area, but it cannot reach the charger at all. If it cannot even touch the target, it cannot proceed to unplug and place it.
“Yuwu™ Lion-ER-35B-A3B” checks each invalid route in turn: red veers toward the left side of the power strip, purple cannot reach the charger, and yellow extends toward a hose in the back. Only the orange trajectory can first reach the charger, unplug it, and then move it to the target position on the right. It answered correctly all 4 times.
The key to this question is connecting the goal, path, and specific operation: the model must consider not only where a path leads, but also whether it can reach the target and complete the task.
Question 3: Across viewpoints—which side of the photographer is the wall art on?
The question presents 3 photos taken inside the same home. The butterfly wall art appears only in the third photo. The model must determine where the artwork was relative to the photographer when the first photo was taken.
Solving this question requires the model to link multiple images and reconstruct the spatial layout. The artwork appears on the living-room wall in the third image; the other 2 images help establish the layout. When taking the first photo, the photographer was standing at the bottom of the stairs, facing the steps, with the living room behind him. The artwork was therefore behind the photographer.
Both comparison models answered “left” and got all 4 attempts wrong. “Yuwu™ Lion-ER-35B-A3B” was correct all 4 times.
Recognizing the artwork is only the basic skill. The difficulty lies in switching to the camera position and facing direction specified by the question to reason about space across viewpoints.
Question 4: Check progress—has this step been completed?
This is a video of a robot performing a task: placing a sushi roll from the table into a pot. The question is whether the first step, “move to the sushi roll,” has been completed at the current point in the video.
The base model incorrectly judged the step “completed” in 3 of its 4 answers, and Qwen3.6-27B made the same mistake in half of its answers. “Yuwu™ Lion-ER-35B-A3B” gave “not yet completed” as its final answer all 4 times.
The key is distinguishing between the robotic arm “approaching the sushi roll” and “having reached the required position.”
An action appearing in the footage does not mean the corresponding task step has been completed. The model needs to observe the change in state caused by the action and check whether the target state has been reached. If it misjudges progress, the robot will carry out subsequent actions prematurely, throwing the entire sequence off course.
The four examples correspond, respectively, to judging camera movement, reasoning about interactions along action trajectories, mapping space across viewpoints, and identifying task status and progress. All are core capabilities for a robot translating language instructions into physical actions.
Embodied large models must not only understand what they see, but also understand physical reality
Multimodal large models that can converse and understand images are now quite common, but “embodied brains” that truly engage with the real physical world and enhance robots’ capabilities remain scarce.
To perform tasks in the real world, robots must do more than recognize what is in an image; they also need to distinguish visual appearances from real three-dimensional space.
“驭物™ Lion-ER-35B-A3B” offers more than a set of impressive scores; it also offers a reusable methodology:
Make capabilities the units of data design, and make evaluation the starting point for data production. From a capability blueprint to a data recipe, and from a single evaluation failure to a new batch of training data, every step has a clear goal, verifiable evidence, and traceable records.
The weights for “驭物™ Lion-ER-35B-A3B” are now open-source on Hugging Face in BF16 Safetensors format, with native compatibility with mainstream inference frameworks including Transformers, vLLM, and SGLang.
Developers can evaluate and adapt the model for understanding spatial relationships and spatial constraints, analyzing scene states, and judging task progress. Using their own images, videos, and tasks, they can explore embodied question answering, scene decision analysis, and integration with robotic systems.
The China Merchants Lion Rock AI Laboratory said the team will subsequently expand training data for embodied agents in areas including tool use, multi-turn interaction, and execution feedback; explore continuous decision-making and recovery from failure in long-horizon tasks; and continue refining this capability-driven data engine to support robots in completing tasks in real-world environments.
Hugging Face model URL:
https://huggingface.co/cmriat/lion-ER-35B-A3B
*This article is republished by QbitAI with permission. The views expressed are solely those of the original author.