This Robot Can Plug In a USB But I Still Can’t

Why is it that every time I try to insert a USB plug it’s backward? Shouldn’t it be right at least half the time by dumb luck? Whatever my problem is, a dextrous new robot doesn’t have it. The robot’s advantage is that its fingertips don’t just feel—they see, too.
Researchers at Northeastern University and (where else?) MIT created the plug-savvy bot. They started with an existing factory-worker robot called Baxter and gave it a pair of pinching fingers. Then on one finger, they added a shrunken-down and adapted version of a sensing technology called GelSight, invented a few years ago.
GelSight creates a precise sense of touch by, essentially, combining it with vision. It uses a small box (below) surrounding a rubber surface that’s coated in metallic paint. A different color of light shines across the surface from each of the box’s four sides: red, green, blue, and white. The rubber acts like the pad of the robot’s finger. As it presses against an object, a camera looking down from the top of the box observes how the four colors of light are reflected by the bending of the metallic pad. Then an algorithm uses that information to create a three-dimensional map of the object’s surface.

“Basically we convert touch information to images,” says lead author Rui Li, a PhD student at MIT. This gives the device an advantage over other kinds of sensors, because it can have a very high resolution—as high as the resolution of the camera built into it.
To test how well GelSight works as a robot’s fingertip, Li and his coauthors asked the newly outfitted Baxter bot to pick up a USB cable dangling from a hook and plug it in. “USB insertion is a very fine manipulation that requires sub-millimeter accuracy, which can be very challenging,” Li says.
Each time the robot picked up the USB cable, it had to feel for the embossed USB symbol on the plug, then use that information to align the plug precisely over the hole. (To treat their sensor gently, the team didn’t make Baxter push the plug all the way down—partway in was good enough.) After the researchers trained the robot, it succeeded on 34 out of 36 attempts to plug in the USB.

Even for robots that don’t spend a lot of time charging their phones, the technology could be useful, Li says. GelSight could help with tasks such as inserting keys, screwing in screws, and any other job that requires grasping an object and lining it up precisely with something else.
Li notes that the group’s robot didn’t actually have to meet the challenge of figuring out which side of the USB plug was up. Because they only equipped the robot with GelSight on one fingertip, they always hung the cord the same way, with the USB symbol facing that finger.
But if the robot had two GelSight fingertips, Li says, the plug could hang in any orientation and the robot would be able to figure it out. “In both cases,” Li says, “the more challenging and critical part is to figure out the relative [position] of the USB plug in the robot’s grippers.” Maybe I can hire one to come hook up my printer.