The newest weapon against mosquitoes: computer vision

Can computers see? The answer is complicated. Ive been following the field of computer vision for decades—ever since Paul Allen and I started dreaming about what you could do with a personal computer—and were only now reaching the point where they can really understand visual inputs. We still have a long way to go, but the ability of computers to see things is already revolutionizing many parts of our lives. It makes autonomous vehicles possible. It’s used to read x-rays quickly and accurately, and it’s what allows a mobile phone to translate street signs from one language to another.

Lately I’ve been especially enthused about a different application (and one my teenage self never would’ve imagined caring about): scanning pictures of mosquitoes.

Mosquitoes are responsible for spreading malaria, which kills more than 600,000 people every year and is a major focus of the Gates Foundation’s health work. Although scientists have learned a lot about them in the past few decades, one challenge has been especially stubborn: telling one mosquito from another. There are around 3,500 different species of them, and many look alike. Even a highly trained entomologist has to examine one for several minutes under a microscope to identify it accurately.

Why do we care about mosquito species? Most importantly, because different species can carry different diseases, and some don’t carry any diseases at all. (The ones that carry malaria belong to the genus Anopheles .) There are other differences too: Some bite people indoors, while others feed outdoors. Some dine at dusk while others take their meals during the day. And only females bite—the blood gives them the energy needed to lay eggs.

All this variation means we need different tools for different mosquitoes. For example, indoor insecticides and bednets work well against species that primarily bite indoors. But for the ones that mainly live and feed outside, you’ll need to take other steps too, such as eliminating the outdoor spaces where they breed.

Fortunately, some novel uses of computer vision are supercharging the process of identification. They’re not only helping us know our opponent, they’re helping us target its weak spots, save more lives, and move even closer to eradicating malaria.

One of the most exciting innovations is called VectorCam—an app that lets someone with minimal training identify mosquito species in a matter of seconds.

VectorCam was developed by Dr. Soumya Acharya and his team of bioengineers at Johns Hopkins University, with support from Uganda’s malaria control program, Makerere University, and the Gates Foundation. Using a smartphone, the VectorCam app, and an inexpensive lens attached to the phone, you simply take a picture of a mosquito and get it identified right away. The app can distinguish among the different species that transmit malaria. It can also determine the sex of the mosquito and, if the insect is a female, whether it has recently fed on blood or developed eggs. And with further refinement, VectorCam could identify species that carry other diseases, like dengue and Zika.

To see the impact a tool like VectorCam can have, consider Uganda, where just 215 people in the entire country are responsible for collecting mosquitoes (often in remote villages), taking them back to a lab, identifying them, and reporting the data to local health systems. The people who do this are known as vector control officers, and they have lots of other responsibilities too, including monitoring for other diseases and working with local hospitals. It’s a huge job.

With VectorCam, a local health worker with minimal training and a mobile phone can do the identifying, freeing up the vector control officer to focus on strategic planning and investigative work. The system is being tested in the Adjumani and Mayuge districts of Uganda, and although we’re still waiting for the final results, it’s already proving useful.

Previous attempts to kill malaria-carrying mosquitoes in these areas had been hampered by the length of time it took to see results. Officials in Adjumani were spraying an insecticide, but they didn’t realize that the local mosquitoes had become resistant to it. They only discovered the problem months later, when they saw that malaria cases in the area hadn’t dropped.

Then, last December, they switched to a different insecticide, and this time, they used VectorCam to monitor the results. Right away, they learned that the spraying worked—the number of mosquitoes responsible for most of the malaria cases plummeted. What’s more, the data from VectorCam showed that as this species died off, others took its place. So the fight goes on, but now the local health teams can fine-tune their efforts because they know immediately what’s working and what isn’t.

VectorCam doesnt just identify mosquitoes, it also streamlines the process of collecting data. In many places, mosquito surveillance still involves filling out paper forms by hand, which then have to be transported and manually entered into computer systems. By the time the data reaches decision-makers, it may be weeks or months out of date. With VectorCam, the data is digitized and aggregated, giving health officials up-to-date information.

There are other promising developments in mosquito surveillance. One has the clever name of HumBug—an app that identifies mosquito species by the sound of their wing beats, captured by the microphone in a smartphone. (It turns out that different species flap their wings at different speeds.) HumBug is still in the early stages, but if it proves out, it could allow for even more automated and continuous monitoring.

Of course, knowing our enemy is only part of the challenge. We also need new and better tools to defeat it. I’ve written a separate post with an update on the effort to eradicate malaria.

In the battle against mosquitoes, were finally getting a clear view of our opponent. Im optimistic that these innovations will bring us closer than ever to a world that is free of malaria.

Quiz answer: The mosquito on the left is from the genus Culex , which can’t carry malaria. But the two others, An. funestus (in the middle) and An. gambiae (on the right), can. An expert could tell the latter two apart by looking at the patterns on their wings, legs, and other body parts. Tricky, isn’t it?