Neural networks to learn to recognize walruses “by sight”

Neural networks to learn to recognize walruses “by sight”

Expert Reports  

The employees of the Nenets State Nature Reserve and the scientists from the Moscow Institute of Physics and Technology (MIPT), with the support of WWF-Russia, will train the artificial intelligence (AI) to analyze data from the camera traps installed in the walrus rookeries, and to recognize pinnipeds in photos as well. To collect the “educational photo material” that will be used to “teach” the neural network, the specialists spent almost three weeks on an island in the Barents Sea.

The Matveyev Island was chosen as a testing ground where the largest Atlantic walrus rookery in the region - up to a thousand animals - is formed every summer under the protection of the Nenets Reserve. Observing this area provides the scientists with an important information about the state of the animals listed in the Russia’s Red Book of Endangered Species. In 2017, to create a safe working environment, WWF helped in equipping an observation and monitoring station on the Matveyev Island.

“Every year, neural networks are increasingly used in studying various species of marine mammals, but no studies of walruses were conducted in this field, at least, in our country,” says Varvara Semyonova, the coordinator of the projects for the conservation of biodiversity in the Arctic carried out under WWF Russia. “In this regard, the Pechora walruses are excellent for practicing this approach as they are a small and very isolated group. This means that the probability of meeting the same walruses in the rookery in other years again is quite high. When we got to know about the initiative of the Nenets Reserve to start this study, we immediately decided to support it. If the neural networks prove to be effective, a similar approach can be applied in other regions of the Arctic, both in the specially protected natural areas and beyond.”

The camera traps are one of the main tools for studying the life in the rookery. The registrars work autonomously all year round and make it possible to understand exactly when the walruses appeared on the shore and leave it, how many animals are in the rookery. However, the problem is that a camera takes tens of thousands of photos in just one season. It is a very time-consuming task to sort them out and analyze such a huge amount of data. It was decided to use modern technologies to help people. The neural network is able to quickly process all the data, leave only the pinnipeds’ photos and sort them out, thereby the Reserve’s team can save hundreds of man-hours.

An even more ambitious task is to teach the computer to recognize individual walruses. To do this, an initial catalogue is now being created - a photo base of animals already seen - on the basis of which the neural network will learn. Each walrus has its own special features (the experts call them “marker points”) that distinguish an individual animal, such as “wrinkles” in the area of the nose and eyes, the scars, scratches, spots on the skin, chips and cracks on the tusks. After analyzing them, the programme is able to “identify” each animal and report on its previous visits to the rookery.

“For example, last year we saw the walrus in the rookery that came once a week. And we understood this while analyzing the photographs for the catalogue. We have been tracking another male animal on the Matveyev Island for 4 years in a row - since 2017,” says Yulia Bogomolova, Head of the Research Department at the Nenets Nature Reserve. “I hope that such identification will also help us monitor the changes in the animal’s physical state and see whether the walrus has grown fat or lost weight, whether it has new scars, see the external signs of diseases, etc. In the future, perhaps, we will monitor how long the animals live.”

The MIPT specialists are responsible for the technical part of the project and direct work with the neural networks. They have already received over 10 thousand images to start training the programme. Andrey Leus, Ph. D. in Engineering Science, running this line of the study, personally joined the expedition this year in order to better assess the tasks and specific peculiarities of the work.

The Reserve’s employees were able to conduct traditional monitoring of the rookery and assess how the walruses live this year on the protected beaches. The conclusions are optimistic as the general preliminary assessment shows that the state of walruses in the Matveyev Island rookery gives us no cause for concern.

“The number of animals that were in the rookery during the observation period, in general, did not differ much from that observed in 2017,” says Yulia Bogomolova. “On certain days, there were up to 700 animals. In addition, this summer, the walruses began to go out on the spit on the Golets Island, just 11 km from the Matveyev Island. In the last 10 years, they appeared here rarely and in very small numbers. Thus, the number of these giant animals in both rookeries is estimated at almost 1,000 walruses.”

Now, the experts are manually analyzing the data obtained on the Matveyev Island. Perhaps, the next field season, they will have much less routine work as the scientists plan to carry out the first tests and conduct an “examination” for the neural network in a few months.