Robots Learn to Grab by Watching Humans
Robots need to know how to grab things, but they don't see the world the same way people do. When a person reaches for a cup, their hand shape tells them what to do. A robot's hand looks different, so the same clues don't work. That mismatch makes it hard for robots to copy human moves. Kids can picture it like a puzzle where the pieces fit only when the shapes match.
To solve this, we built a huge collection of 3D shape guides called HPAD. It holds 2,137 items spread over 80 groups, each tagged with both a person's guess and a robot's verified label. The set includes 13 basic move types and more than 27,000 pairings of objects with actions.
Our method, named ART, first runs a trust filter to drop wrong human clues. Then it rebuilds the clues step by step and smooths them with a network trick that matches the robot's shape layout. This fixes the shape layout problem between human and robot views.
We tested ART on a six‑move joint arm with a two‑finger hand, trying 12 objects and four move types across 2,400 tries. The system succeeded 86.5 percent of the time, showing that the idea works in real life.