Wearable device takes the sense of touch to the metaverse

A research team co-led by City University of Hong Kong (CityU) has developed a wearable tactile rendering system, which can mimic the sensation of touch with high spatial resolution and a rapid response rate.

The team demonstrated its application potential in a braille display, adding the sense of touch in the metaverse for functions such as virtual reality shopping and gaming, and potentially facilitating the work of astronauts, deep-sea divers and others who need to wear thick gloves.

“We can hear and see our families over a long distance via phones and cameras, but we still cannot feel or hug them. We are physically isolated by space and time, especially during this long-lasting pandemic. Although there has been great progress in developing sensors that digitally capture tactile features with high resolution and high sensitivity, we still lack a system that can effectively virtualise the sense of touch that can record and play back tactile sensations over space and time,” said Dr Yang Zhengbao, an Associate Professor at CityU who co-led the study.

In collaboration with Tencent’s Robotics X Laboratory, the researchers developed a novel electrotactile rendering system for displaying various tactile sensations with high spatial resolution and a rapid response rate. Their findings were published in the journal Science Advances.

Existing techniques to reproduce tactile stimuli can be classified into two categories: mechanical and electrical stimulation. By applying a localised mechanical force or vibration on the skin, mechanical actuators can elicit stable and continuous tactile sensations. However, they tend to be bulky, limiting the spatial resolution when integrated into a portable or wearable device. Electrotactile stimulators, however, evoke touch sensations in the skin at the location of the electrode by passing a local electric current through the skin, and can be light and flexible while offering higher resolution and a faster response. But most of them rely on high voltage direct-current (DC) pulses (up to hundreds of volts) to penetrate the stratum corneum, the outermost layer of the skin, to stimulate the receptors and nerves, which poses a safety concern. The tactile rendering resolution also needs to be improved.

The electrotactile actuator developed by CityU researchers is very thin and flexible and can be easily integrated into a finger cot. This fingertip wearable device can display different tactile sensations, such as pressure, vibration, and texture roughness in high fidelity. Instead of using DC pulses, the researchers developed a high-frequency alternating stimulation strategy and succeeded in lowering the operating voltage under 30 V, to make the tactile rendering safe and comfortable. The researchers proposed a novel super-resolution strategy that can render tactile sensation at locations between physical electrodes, instead of only at the electrode locations. This increases the spatial resolution of their stimulators by more than three times (from 25 to 105 points), so the user can feel more realistic tactile perception.

“Our new system can elicit tactile stimuli with both high spatial resolution (76 dots/cm²), similar to the density of related receptors in the human skin, and a rapid response rate (4 kHz),” said Lin Weikang, a PhD student at CityU, who made and tested the device.

The researchers ran different tests to show various application possibilities of this new wearable electrotactile rendering system. For example, they proposed a new Braille strategy that is easier for people with a visual impairment to learn. The proposed strategy breaks down the alphabet and numerical digits into individual strokes in the same order that they are written; by wearing the new electrotactile rendering system on a fingertip, the user can recognise the alphabet presented by feeling the direction and the sequence of the strokes with the fingertip sensor. “This would be particularly useful for people who lose their eyesight later in life, allowing them to continue to read and write using the same alphabetic system they are used to, without the need to learn the whole Braille dot system,” Yang said.

The new system is well-suited for VR/AR applications and games, adding the sense of touch to the metaverse. The electrodes can be flexible and scalable to cover larger areas, such as the palm. The researchers demonstrated that a user can virtually sense the texture of clothes in a virtual fashion shop. The user also experienced an itchy sensation in the fingertips when being licked by a VR cat. When stroking a virtual cat’s fur, the user can feel a variance in the roughness as the strokes change direction and speed. The system can also be useful in transmitting fine tactile details through thick gloves. The team integrated the thin, light electrodes of the electrotactile rendering system into flexible tactile sensors on a safety globe. The tactile sensor array captures the pressure distribution on the exterior of the glove and relays the information to the user in real time through tactile stimulation. In the experiment, the user could quickly locate a tiny steel washer 1 mm in radius and 0.44 mm thick based on the tactile feedback from the glove with sensors and stimulators. This shows the system’s potential in enabling high-fidelity tactile perception, which is unavailable to astronauts, firefighters and others who need to wear thick protective suits or gloves.

“We expect our technology to benefit a broad spectrum of applications, such as information transmission, surgical training, teleoperation, and multimedia entertainment,” Yang said.

Image credit: Robotics X Lab and City University of Hong Kong