REACT lets a Unitree G1 handle close contact from its own camera

Close physical contact is a hard perception problem for a humanoid. The moment a person reaches in for a handshake or hug, their body fills the robot’s camera, limbs disappear from view and a conventional pose estimator can lose the very state the controller needs.

REACT, a new Carnegie Mellon University and Keio University research system, tackles that failure mode on a Unitree G1. The researchers report 31 successes across 50 staged hardware trials spanning five interactions, with the policies transferred from simulation without real-world fine-tuning. The robot used onboard sensing and was not teleoperated during the trials.

That is meaningful real-robot evidence. It is also a 62% aggregate result in a narrow, cooperative setup — not evidence that a humanoid is ready to support people safely in a home, hospital or workplace.

Contact becomes the training signal

Most motion-retargeting pipelines ask a robot to copy a person’s pose. That is often enough for a dance or isolated gesture, but it can miss what defines an interaction: which body parts should touch, where and when.

REACT begins with human–human motion-capture sequences. A vision-language model reviews multi-view renderings and filters candidate contacts, trying to separate meaningful events — palm meeting palm in a high-five, for example — from body parts that merely pass close together. An optimisation stage then retargets both motion and those selected contacts to the G1 and a simulated human partner.

A privileged teacher policy learns the interaction with full access to the partner’s simulated state. A student policy is then trained to reproduce the behaviour using only egocentric depth and the robot’s own motion history. An auxiliary generative objective asks the student to reconstruct and anticipate the partner’s motion, giving its internal representation a way to carry interaction state through occlusion.

The paper’s simulation results support the contact-centred design. In its retargeting comparison, the full system reached 59.4% contact recall, versus 22.1% for OmniRetarget and 0.42% for GMR. Those are paper-defined simulation metrics against the authors’ annotations; they are not physical-robot success rates.

The real G1 result is 31 out of 50

The hardware evaluation is unusually useful because the authors report failures as well as successful clips. Each interaction was attempted ten times on a physical G1:

- high-five: 5/10 - hug: 8/10 - handshake: 5/10 - link arms: 5/10 - pat on the back: 8/10

For high-fives and handshakes, success meant reacting to the partner’s raised or lowered hand and bringing the robot’s hand to within roughly 15 centimetres. For the other three interactions, the partner stood still at varied positions and success meant that the G1 approached and completed the interaction.

Those criteria make the result easier to interpret. A high-five counted without requiring precise palm contact, while three of the five tasks used a stationary partner. The trials show reactive, autonomous control in a structured laboratory setting. They do not establish performance with people who move unpredictably, resist contact or need physical support.

The project also shows a roughly 30-second rollout containing three consecutive hugs without a reset. That demonstrates recovery and repetition within one staged interaction. It is not a long-duration reliability test.

“Onboard depth” needed a learned workaround

REACT does not use external motion capture or a third-person tracking system at deployment. But “onboard sensing” deserves a closer look.

The G1’s Intel RealSense D435 stereo depth stream became noisy and lost the person’s silhouette while the robot moved. Instead of using that depth output, the researchers ran Depth Anything 3 on the onboard RGB image and fed the estimated metric depth to the policy through an onboard TensorRT/DeepStream pipeline.

That is a sensible engineering response to a real sensor limitation. It also means the result depends on a learned monocular depth model and the conditions represented in its training and the team’s simulator. Lighting, clothing, reflective surfaces and unusual body shapes remain relevant sources of error.

The paper’s simulated comparison against an external HMR2.0 pose-estimation pipeline is consistent with the core argument: close-range occlusion caused the baseline’s contact recall to collapse across all five interactions. REACT’s end-to-end depth policy performed substantially better in that benchmark. Independent replication is still needed, and the comparison was conducted in simulation rather than as a matched hardware trial.

Autonomous does not mean general

The five hardware behaviours were evaluated as separate interaction policies. The researchers also describe a vision-language-model selector that can search for a partner and switch between these skills, but the paper presents that composition qualitatively rather than as the same 50-trial hardware benchmark.

REACT therefore should not be read as one general policy that understands arbitrary social situations. It is a collection of learned, closed-loop skills for five named interactions. The human participant is cooperative, the environment is controlled and the robot’s hands are treated as rigid end effectors; finger actuation and force-sensitive grasp shaping are out of scope.

The safety evidence is similarly preliminary. Training includes a force-regulation reward intended to keep simulated contact within a target band of 5–50 newtons. The paper does not report an independent safety assessment, injury-risk study or certification, and it does not publish a distribution of measured contact forces from the 50 physical trials.

What this changes for human-facing robots

The important contribution is not that a G1 can hug. It is that the control stack treats contact and hidden partner motion as first-class learning problems, then tests the resulting policies on hardware with a disclosed trial count.

That direction matters for Australian labs and organisations studying cobotics, rehabilitation and care. Human-facing robots will need to keep acting when a person occludes the camera, but they will also need stronger evidence than a successful clip: repeated trials with diverse participants, measured contact forces, active gaze, stop behaviour, fault recovery and formal human-factors review.

REACT is a credible research step toward that evidence. Its current numbers also show why deployment claims would be premature. Half of the high-five, handshake and link-arms trials did not meet the authors’ own success criteria. The camera cannot actively reposition to recover a lost view, the repertoire is short and upper-body focused, and the public repository says training code is expected in early November 2026 rather than available now.

The right reading is neither “just a demo” nor “safe social robot”. REACT demonstrates that semantic contact supervision and predictive onboard perception can cross from simulation to a real Unitree G1. What remains is the harder work of reliability, dexterity, safety validation and independent reproduction.

PHASE reviewed the paper, submission record, project page and public repository on 24 October 2026. This is an analysis of public research, not a hands-on test.