Tactile sensor gives robots dexterity

Researchers have equipped a robot with a tactile sensor that lets it grasp a USB cable draped freely over a hook and insert it into a USB port.

The sensor is an adaptation of a technology called GelSight, developed by the lab of Edward Adelson - the John and Dorothy Wilson Professor of Vision Science at MIT - and first described in 2009. The new sensor is not as sensitive as the original GelSight sensor, which could resolve details on the micrometre scale. It is smaller - small enough to fit on a robot’s gripper - and its processing algorithm is faster, so it can give the robot feedback in real time.

Industrial robots are capable of remarkable precision when the objects they are manipulating are perfectly positioned in advance. But according to Robert Platt, an assistant professor of computer science at Northeastern University and the research team’s robotics expert, for a robot taking its bearings as it goes, this type of fine-grained manipulation is unprecedented.

The MIT team - which consists of Adelson; first author Rui Li, a PhD student; Wenzhen Yuan, a master’s student; and Mandayam Srinivasan, a senior research scientist in the Department of Mechanical Engineering - designed and built the sensor. Platt’s team at Northeastern developed the robotic controller and conducted the experiments.

Whereas most tactile sensors use mechanical measurements to gauge mechanical forces, GelSight uses optics and computer-vision algorithms.

A GelSight sensor - both the original and the new, robot-mounted version - consists of a slab of transparent, synthetic rubber coated on one side with metallic paint. The rubber conforms to any object it’s pressed against and the metallic paint evens out the light-reflective properties of diverse materials, making it much easier to make precise optical measurements.

In the new device, the gel is mounted in a cubic plastic housing, with just the paint-covered face exposed. The four walls of the cube adjacent to the sensor face are translucent and each conducts a different colour of light - red, green, blue or white - emitted by light-emitting diodes at the opposite end of the cube. When the gel is deformed, light bounces off the metallic paint and is captured by a camera mounted on the same cube face as the diodes.

From the different intensities of the different-coloured light, the algorithms developed by Adelson’s team can infer the three-dimensional structure of ridges or depressions of the surface against which the sensor is pressed.

Although there are several ways of measuring human tactile acuity, one is to determine how far apart two small bumps need to be before a subject can distinguish them just by touching; the answer is usually about a millimetre. By that measure, even the lower resolution, robot-mounted version of the GelSight sensor is about 100 times more sensitive than a human finger. In Platt’s experiments, a Baxter robot from MIT spinout Rethink Robotics was equipped with a two-pincer gripper, one of whose pincers had a GelSight sensor on its tip. Using conventional computer-vision algorithms, the robot identified the dangling USB plug and attempted to grasp it. It then determined the position of the USB plug relative to its gripper from an embossed USB symbol. Although there was a 3-mm variation in each of two dimensions where the robot grasped the plug, it was still able to insert it into a USB port that tolerated only about a millimetre’s error.