Faster reaction time helps drones dodge obstacles

Using a novel type of camera, researchers from the University of Zurich (UZH) have demonstrated a flying robot that can detect and avoid fast-moving objects — a skill which today’s drones are currently lacking.

Although many flying robots are equipped with cameras that can detect obstacles, it typically takes from 20 to 40 ms for the drone to process the image and react. This is not quick enough to avoid a bird or another drone, or even a static obstacle when the drone itself is flying at high speed. This can be a problem when drones are used in unpredictable environments, or when there are many of them flying in the same area.

In order to solve this problem, UZH researchers equipped a quadcopter (a drone with four propellers) with special cameras and algorithms that reduced its reaction time down to a few milliseconds — enough to avoid a ball thrown at it from a short distance. The results, published in the journal Science Robotics, can make drones more effective in situations such as the aftermath of a natural disaster.

“For search and rescue applications, such as after an earthquake, time is very critical, so we need drones that can navigate as fast as possible in order to accomplish more within their limited battery life,” said Davide Scaramuzza, who leads the Robotics and Perception Group at the UZH as well as the NCCR Robotics Search and Rescue Grand Challenge. “However, by navigating fast drones are also more exposed to the risk of colliding with obstacles, and even more if these are moving.” The researchers realised that a novel type of camera, called an event camera, was a perfect fit for this purpose.

Video credit: UZH.

Traditional video cameras, such as the ones found in every smartphone, work by regularly taking snapshots of the whole scene. This is done by exposing the pixels of the image all at the same time. This way, though, a moving object can only be detected after all the pixels have been analysed by the onboard computer. Event cameras, on the other hand, have smart pixels that work independently of each other. The pixels that detect no changes remain silent, while the ones that see a change in light intensity immediately send out the information. This means that only a tiny fraction of the all pixels of the image will need to be processed by the onboard computer, therefore speeding up the computation a lot.

Event cameras are a recent innovation, and existing object-detection algorithms for drones do not work well with them. So the researchers had to invent their own algorithms that collect all the events recorded by the camera over a very short time, then subtract the effect of the drone’s own movement — which typically accounts for most of the changes in what the camera sees.

Scaramuzza and his team first tested the cameras and algorithms alone. They threw objects of various shapes and sizes towards the camera, and measured how efficient the algorithm was in detecting them. The success rate varied between 81 and 97%, depending on the size of the object and the distance of the throw, and the system only took 3.5 ms to detect incoming objects.

Then the most serious test began: putting cameras on an actual drone, flying it both indoors and outdoors, and throwing objects directly at it. The drone was able to avoid the objects — including a ball thrown from a 3 m distance and travelling at 10 m/s — more than 90% of the time. When the drone ‘knew’ the size of the object in advance, one camera was enough. When it had to face objects of varying size, two cameras were used to give it stereoscopic vision.

The camera captures the approaching ball. Image credit: UZH.

According to Scaramuzza, these results show that event cameras can increase the speed at which drones can navigate by up to 10 times, thus expanding their possible applications.

“One day drones will be used for a large variety of applications, such as delivery of goods, transportation of people, aerial filmography and, of course, search and rescue,” he said. “But enabling robots to perceive and make decisions faster can be a game changer for … other domains where reliably detecting incoming obstacles plays a crucial role, such as automotive, good delivery, transportation, mining and remote inspection with robots.”

In the future, the team aims to test this system on an even more agile quadrotor. According to PhD student Davide Falanga, the primary author of the study, “Our ultimate goal is to make one day autonomous drones navigate as good as human drone pilots. Currently, in all search and rescue applications where drones are involved, the human is actually in control. If we could have autonomous drones navigate as reliably as human pilots we would then be able to use them for missions that fall beyond line of sight or beyond the reach of the remote control.”

Top image caption: The drone is able to successfully dodge — even if the ball is approaching it from a distance of 3 m at 10 m/s. Image credit: UZH.

Please follow us and share on Twitter and Facebook. You can also subscribe for FREE to our weekly newsletter and bimonthly magazine.