正文 · AI 翻译
IT Home reported on November 10 that Ant Lingbo Technology open-sourced LingBot-Touch, a foundational tactile model, today, adding a key sense of touch to robot brains.
According to the official introduction obtained by IT Home, LingBot-Touch was pretrained on nearly 5,000 hours of real tactile data. It covers 3 mainstream sensing principles—magnetic tactile, vision-based tactile and piezoresistive—and converts their different signals into a unified tactile representation, allowing accumulated “feel” to be reused across sensors, dexterous hands, models and tasks. In 26 tests on physical robots, adding this representation raised the average task success rates of 4 action models by about 14 to 26 percentage points compared with operating without touch.
Previously, Ant Lingbo’s LingBot series of models had built a full-stack technical system spanning high-precision spatial perception, interaction with highly dynamic environments and generalizable dexterous manipulation, enabling robots to see more clearly, reason more effectively and act more deftly. LingBot-Touch now extends the general-purpose brain’s perception to the “fingertips,” making robots “more sensitive to touch” and further improving the capabilities of an embodied-native general-purpose brain.
Once robots can see clearly, why do they still need to “feel more sensitively”?
In the real physical world, every action a robot takes requires it to sense changes in objects and its surroundings, then adjust its next action. In many fine manipulation tasks, seeing where an object is and moving a hand to the right place does not mean the task will go smoothly. After contact, vision alone often cannot determine whether the applied force is appropriate or whether the object is slipping.
Take unscrewing a bottle cap: depth information can help the fingers line up with the cap. But once they start turning, the cap may have loosened, or the fingertips may be slipping along its side. The 2 situations look similar on camera but require different adjustments. The location of fingertip contact, the amount of force and how they change become important cues for the action model; fingers and objects often block these details from view.
To verify how tactile representations help with real-world manipulation, the team conducted 26 tests on physical robots using 2 dexterous hands, Sharpa Wave and XHand, covering fine tasks such as turning book pages, unscrewing bottle caps and identifying mahjong tiles. With LingBot-Touch, the average task success rates of 4 action models—LingBot-VLA, π0.5, Diffusion Policy and T-Rex—rose by about 14 to 26 percentage points compared with operating without touch.
In the task where T-Rex controlled Sharpa Wave to unscrew a bottle cap, the success rate was 56.25% without touch and 62.5% when raw sensor signals were fed directly into the model. With LingBot-Touch’s tactile representation, it reached 90%.
This series of tests on physical robots shows that high-quality tactile input can significantly improve robots’ success rates in performing actions.
A unified tactile representation reusable across dexterous hands and models
Fitting a robot with tactile sensors does not mean it can use touch effectively. Commercial tactile sensors are still at a relatively early stage of development overall, and vision-based tactile, magnetic tactile and piezoresistive approaches each have strengths and limitations. Multiple approaches may coexist for a long time to serve different costs and use cases.
More challenging is that the same contact produces different raw signals under different sensing principles and contact-point layouts. If a model learns to interpret only 1 type, its existing data and tactile experience may be difficult to reuse after a sensor or dexterous hand is changed, requiring new data collection, training and adaptation.
The team’s approach is to free tactile understanding from dependence on the physical signals of a particular sensor, so data from different sources can be used together for training and accumulated tactile capabilities can be reused across dexterous hands and action models.
To do this, LingBot-Touch uses separate encoding inputs for each sensing principle, first processing the different raw signals and then mapping contact points with different layouts into a unified fingertip coordinate space. A shared model learns contact location, force and how they change over time, producing unified tactile tokens—a tactile representation for use by different action models.
Under this architecture, data collected from people wearing tactile gloves and from teleoperated dexterous hands can be used together for training. Different action models can also use the trained model to extract contact features. The architecture leaves room to add new types of tactile sensors in the future.
The team further tested reuse of tactile capabilities across hardware on XHand. The magnetic tactile sensor model used by XHand was absent from the pretraining data; all the magnetic tactile experience the model learned came from data collected from people wearing gloves. After switching to this dexterous hand, the team kept LingBot-Touch’s parameters unchanged and trained only the action model using demonstrations of XHand operations.
LingBot-Touch provides a general-purpose perceptual capability between tactile sensors and action models, rather than a dedicated module for a particular piece of hardware or policy. This round of testing used 4 different action models and was not tied to the team’s own LingBot-VLA model. Sensor and dexterous-hand manufacturers can choose hardware based on cost and task requirements, while teams building complete robots can retain their own action policies, reducing repeated work to redevelop tactile capabilities when changing hardware and algorithm combinations.
To balance perceptual capability with on-device computing capacity, the team offers models in 2 sizes. All 26 tests on physical robots used the lightweight LingBot-Touch-Tiny, which has about 5.3 million parameters. When integrated with LingBot-VLA, π0.5 and T-Rex, it increased model parameter counts by only about 0.13% to 0.16%.
The larger LingBot-Touch-Small showed potential for further improvement in perception tests: when pretraining data grew to nearly 5,000 hours, the model continued to benefit from additional data, indicating room for continued gains through scaling tactile pretraining.
Ant Lingbo said that, as general-purpose large models advance rapidly, the release of LingBot-Touch reflects the team’s continued commitment to and confidence in an “embodied-native” approach. Unlike multimodal paradigms designed for digital information, robots operate in the real physical world: information comes from sensors and actions affect the environment. They need to understand spatial perceptual information such as depth, as well as process contact-related modalities such as touch, force and temperature. The quality of modal inputs largely determines the quality of action outputs.
References
Project homepage
Advertising statement: External links in this article (including but not limited to hyperlinks, QR codes and passcodes) are provided to convey more information and save time in selecting information. The results are for reference only. All IT Home articles containing external links include this statement.