Stretch-sensing glove captures hand poses in real time

Capturing interactive hand poses in real time and with realistic results is a well-examined problem in computing, particularly human-centred computing and motion capture technology. Computer scientists have now advanced this area of research by developing a user-friendly, stretch-sensing data glove to capture real-time, interactive hand poses with precision.

Human hands are complex — an intricate system of flexors, extensors and sensory capabilities serving as our primary means to manipulate physical objects and communicate with one another. The accurate motion capture of hands is relevant and important for many applications, such as gaming, augmented and virtual reality domains, robotics and biomedical industries.

Computer scientists from ETH Zurich and New York University set out to overcome some persisting challenges in the replication of accurate hand poses. In this work, they addressed hurdles such as capturing the hand motions in real time in a variety of environments and settings, as well as using only user-friendly equipment and an easy-to-learn approach for set-up.

The researchers utilised a silicone compound in the shape of a hand equipped with 44 stretch sensors and attached this to a glove made of soft, thin fabric. To reconstruct the hand pose from the sensor readings, the researchers use a data-driven model that exploits the layout of the sensor itself. The model is trained only once, with training data gathered using an inexpensive, off-the-shelf hand pose reconstruction system.

The main advantage of their stretch-sensing gloves, the researchers said, is that they do not require a camera-based set-up — or any additional external equipment — and could begin tracking hand poses in real time with only minimal calibration.

“To our best knowledge, our gloves are the first accurate hand-capturing data gloves based solely on stretch sensors,” said Oliver Glauser, a PhD student at ETH Zurich and lead author of the work. “The gloves are soft and thin, making them very comfortable and unobtrusive to wear, even while having 44 embedded sensors. They can be manufactured at a low cost with tools commonly available in fabrication labs.”

The researchers demonstrated that their gloves are successful in accurately computing hand poses in real time, even while the user is holding a physical object and in conditions such as low lighting. When compared to two commercial glove products, the researchers’ gloves received the lowest error return for all but one hand pose.

In future work, the team intends to explore how a similar sensor approach could be used to track a whole arm to get the global position and orientation of the glove, or perhaps even a full body suit. Currently the researchers have fabricated medium-sized gloves, and they would like to expand to other sizes and shapes.

Image caption: The stretch-sensing soft glove captures hand poses in real time. It functions in diverse and challenging settings. Image© ETH Zurich.

Please follow us and share on Twitter and Facebook. You can also subscribe for FREE to our weekly newsletter and bimonthly magazine.