Imagine seeing the world through an insect's eyes — traversing rocky, uneven terrain and swooping into hard-to-reach spaces. Engineers at the University of Washington announced Wednesday this soon may be a reality, via a wireless camera small enough to mount on the back of a live beetle. In other words, a 'GoPro for insects.'
Funded through a Microsoft Fellowship, as well as a $980,000 grant from the National Science Foundation, the engineers claim to have developed the first fully wireless mechanically steerable vision system in a form factor tiny enough to ride aboard an insect and stream video to a smartphone at 1 to 5 frames per second. The entire system weighs about 250mg — about one-tenth the weight of a playing card, and has enough battery power to record images for around six hours.
The camera sits on a mechanical arm that can pivot 60 degrees to enable it to pan from left to right and capture high-resolution, panoramic shots or track a moving object. The camera and arm are controlled via Bluetooth from a smartphone from a distance up to about 395 ft (120 m) away, just a little longer than a football field.Â
Until now, wireless vision has not been possible for small robots or insects, said Shyam Gollakota, a UW associate professor in the Paul G. Allen School of Computer Science & Engineering, who led the study.
Why beetles?
The researchers chose to attach the removable system to the backs of two different types of beetles — a death-feigning beetle and a Pinacate beetle — as similar beetles have been known to be able to carry loads heavier than half a gram, they said.
"We made sure the beetles could still move properly when they were carrying our system," said co-lead author Ali Najafi, a UW doctoral student in electrical and computer engineering. "They were able to navigate freely across gravel, up a slope and even climb trees."
Vikram Iyer, a UW doctoral student in electrical and computer engineering and co-lead of the study, added that they gradually added weight to make sure it didn't affect their motion. "While carrying the camera, they were still able to easily traverse obstacles like rocks that were larger than them. We were also able to easily detach our camera system after finishing our experiments and saw that the beetles lived for over a year."
This is far beyond the capabilities of small robots. It opens up the possibility of using insects with sensors to monitor conditions for smart farming, locate colonies of insects for pest control and explore things like pipes or other confined spaces.Vikram IyerUW doctoral student in electrical and computer engineering and co-lead of the study
This isn't the first time the UW researchers introduced a method for insects in flight. In December 2018 they announced they had developed the first flying wireless platform with a power source both strong and light enough to attach to a bumblebee. It included sensors, wireless communication and location trackers. The engineers referred to it as Living IoT. The sensors required only a tiny rechargeable battery that could last for seven hours of flight. The whole sensor package weighed only 102 mg, or about the weight of seven grains of uncooked rice. The engineers could track the bees' location, and the sensors could monitor temperature, humidity and light intensity.
Iyer said they have never received any backlash from environmentalists on experimenting with bees, but did actually get some interest from beekeepers who preferred the idea of developing systems that use live insects rather than trying to engineer artificial pollinators.
Prior to this, the UW engineers unveiled RoboFly, a robo-insect slightly heavier than a toothpick that was powered by an invisible laser beam pointed at a photovoltaic cell attached above the robot, which converted the laser light into enough electricity to operate its wings. But with its tiny battery, the most it could do was take off and land.
New uses for small-scale wireless vision
The dramatic reduction in the size, weight and power of traditional vision systems extends the use of cameras to new applications that were not possible before, said Gollakota.
"Vision is so important for communication and for navigation. ... Having the ability to capture and stream visual data as the insect moves around can be extremely beneficial to the agricultural and farming industry, as well as industrial IoT and exploring novel environments," he said.
The team is releasing their code for this project for anyone to use.
"Live insects like beetles can walk around for hours and even climb trees while carrying the camera. Since this is far beyond the capabilities of small robots, it opens up the possibility of using insects with sensors to monitor conditions for smart farming, locate colonies of insects for pest control and explore things like pipes or other confined spaces," said Iyer.
Gollakota also claims that precision agriculture companies, which are designing and deploying sensors for farms, will be interested in this technology "since we are providing the ability to capture information below the canopy while enabling mobility."
A power lesson in biology
Typical small cameras, such as those used in smartphones, use a lot of power to capture wide-angle, high-resolution photos, and that doesn't work at the insect scale. While the cameras themselves are lightweight, the batteries they need to support them make the overall system too big and heavy for insects — or insect-size robots — to lug around. So the team took a lesson from biology. Â
"Similar to cameras, vision in animals requires a lot of power," said co-author Sawyer Fuller, a UW assistant professor of mechanical engineering. "It's less of a big deal in larger creatures like humans, but flies are using 10% to 20% of their resting energy just to power their brains, most of which is devoted to visual processing. To help cut the cost, some flies have a small, high-resolution region of their compound eyes. They turn their heads to steer where they want to see with extra clarity, such as for chasing prey or a mate. This saves power over having high resolution over their entire visual field." Â
To mimic an insect's vision, the researchers used a tiny, ultra-low-power black-and-white camera that can sweep across a field of view with the help of a mechanical arm. The arm moves when the team applies a high voltage, which makes the material bend and move the camera to the desired position. Unless the team applies more power, the arm stays at that angle for about a minute before relaxing back to its original position. This is similar to how people can keep their head turned in one direction for only a short period of time before returning to a more neutral position.Â
"One advantage to being able to move the camera is that you can get a wide-angle view of what's happening without consuming a huge amount of power by moving the whole robot," said Iyer.
As a result, this camera can collect visual information using 26 to 84 times less energy, said Gollakota.
"We added a small accelerometer to our system to be able to detect when the beetle moves. It only captures images during that time," Iyer said. "If the camera is just continuously streaming without this accelerometer, we could record one to two hours before the battery died. With the accelerometer, we could record for six hours or more, depending on the beetle's activity level.Â
The team hopes that future versions of the camera will require even less power and be battery free, potentially even solar-powered.
