Turning off the TV with a cup of tea: join the remote control revolution

We’ve all found ourselves, at one point or another, on that seemingly endless search for the TV remote, only to discover once we find it that the batteries aren’t working anyway. It’s an experience which could soon be at an end, with Lancaster University researchers having developed gesture control technology that allows body movement, or movement of objects, to be used to interact with screens.

Developed by Professor Hans Gellersen and PhD student Christopher Clarke, ‘Matchpoint’ technology requires only a simple webcam and works by displaying moving targets that orbit a small circular widget in the corner of the screen. These targets correspond to different functions — such as volume, changing channel or viewing a menu. When selecting volume adjustment or channel selection, sliders appear.

The user synchronises the direction of movement of the target with their hand, head or an object to achieve what researchers call ‘spontaneous spatial coupling’. Thus, the user moves their hand, head or object in the required direction indicated by the slider to change the volume or to find the desired channel.

“Spontaneous spatial coupling is a new approach to gesture control that works by matching movement instead of asking the computer to recognise a specific object,” explained Clarke.

Unlike existing gesture control technology, the software does not look for a specific body part it has been trained to identify, such as a hand; instead, it looks for rotating movement. This means it does not require calibration, or prior knowledge of objects, thus providing much more flexibility and ease for the user.

As well as televisions, the technology can also be used with other screens. For example, YouTube tutorials, such as mending bikes or baking cakes, could be easily paused and rewound on tablet computers without users having to put down tools or mixing bowls.

“Our method allows for a much more user-friendly experience, where you can change channels without having to put down your drink or change your position; whether that is relaxing on the sofa or standing in the kitchen following a recipe,” said Clarke.

The technology can also be utilised for interactive whiteboards, with multiple pointers allowing more than one user to point at drawings or pictures. Matchpoint also allows users to manipulate images on whiteboards by using two hands to zoom in and out, and to rotate images.

The researchers believe Matchpoint could be particularly beneficial for people who are unable to use traditional pointers, such as remote controls, mice and keyboards. They noted that users do not need to learn specific commands to activate different functions, and they have the option to decouple at will.

In addition to short-term couplings, users can also link stationary objects to controls, which even when left for prolonged periods will retain their control function. For example, a mug sitting on a table could change a track on a music player when moved left or right, and a rolling toy car could be used to adjust volume. Objects can lose their coupling with controls simply by removing them from the camera’s field of view.

“Everyday objects in the house can now easily become remote controls, so there are no more frantic searches for remote controls when your favourite program is about to start on another channel, and now everyone in the room has the ‘remote’,” said Clarke. “You could even change the channel with your pet cat.”

The technology has been described in the paper ‘Matchpoint: Spontaneous spatial coupling of body movement for touchless pointing’, to be presented at the 30th ACM Symposium on User Interface Software and Technology (UIST) in Quebec from 22–25 October.

Image caption: Lancaster University researcher Christopher Clarke selects a channel to watch by using his mug as a remote control. He moves his drink left or right until finding the station he wants to watch. Image credit: Lancaster University.