of 15
PERSPECTIVE
Perspectives in machine learning for wildlife
conservation
Devis Tuia
1,17
, Benjamin Kellenberger
1,17
, Sara Beery
2,17
,
Blair R. Costelloe
3,4,5,17
, Silvia Zuf
fi
6
, Benjamin Risse
7
,
Alexander Mathis
8
, Mackenzie W. Mathis
8
, Frank van Langevelde
9
,
Tilo Burghardt
10
, Roland Kays
11,12
, Holger Klinck
13
, Martin Wikelski
3,4
,
Iain D. Couzin
3,4,5
, Grant van Horn
13
, Margaret C. Crofoot
3,4,5
,
Charles V. Stewart
14
& Tanya Berger-Wolf
15,16
Inexpensive and accessible sensors are accelerating data acquisition in animal ecology. These
technologies hold great potential for large-scale ecological understanding, but are limited by
current processing approaches which inef
fi
ciently distill data into relevant information. We
argue that animal ecologists can capitalize on large datasets generated by modern sensors by
combining machine learning approaches with domain knowledge. Incorporating machine
learning into ecological work
fl
ows could improve inputs for ecological models and lead to
integrated hybrid modeling tools. This approach will require close interdisciplinary colla-
boration to ensure the quality of novel approaches and train a new generation of data
scientists in ecology and conservation.
A
nimal diversity is declining at an unprecedented rate
1
. This loss comprises not only
genetic, but also ecological and behavioral diversity, and is currently not well understood:
out of more than 120,000 species monitored by the IUCN Red List of Threatened Species,
up to 17,000 have a
Data de
fi
cient
status
2
. We urgently need tools for rapid assessment of
wildlife diversity and population dynamics at large scale and high spatiotemporal resolution,
from individual animals to global densities. In this
Perspective,
we aim to build bridges across
ecology and machine learning to highlight how relevant advances in technology can be leveraged
to rise to this urgent challenge in animal conservation.
https://doi.org/10.1038/s41467-022-27980-y
OPEN
1
School of Architecture, Civil and Environmental Engineering, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland.
2
Department of
Computing and Mathematical Sciences, California Institute of Technology (Caltech), Pasadena, CA, USA.
3
Max Planck Institute of Animal Behavior,
Radolfzell, Germany.
4
Centre for the Advanced Study of Collective Behaviour, University of Konstanz, Konstanz, Germany.
5
Department of Biology,
University of Konstanz, Konstanz, Germany.
6
Institute for Applied Mathematics and Information Technologies, IMATI-CNR, Pavia, Italy.
7
Computer Science
Department, University of Münster, Münster, Germany.
8
School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland.
9
Environmental Sciences Group, Wageningen University, Wageningen, Netherlands.
10
Computer Science Department, University of Bristol, Bristol, UK.
11
Department of Forestry and Environmental Resources, North Carolina State University, Raleigh, NC, USA.
12
North Carolina Museum of Natural Sciences,
Raleigh, NC, USA.
13
Cornell Lab of Ornithology, Cornell University, Ithaca, NY, USA.
14
Department of Computer Science, Rensselaer Polytechnic Institute,
Troy, NY, USA.
15
Translational Data Analytics Institute, The Ohio State University, Columbus, OH, USA.
16
Departments of Computer Science and
Engineering; Electrical and Computer Engineering; Evolution, Ecology, and Organismal Biology, The Ohio State University, Columbus, OH, USA.
17
These
authors contributed equally: Devis Tuia, Benjamin Kellenberger, Sara Beery, Blair R. Costelloe.
email:
devis.tuia@ep
fl
.ch
NATURE COMMUNICATIONS
| (2022) 13:792 | https://doi.org/10.1038/s41467-022-27980-y | www.nature.com/naturecommunications
1
1234567890():,;
How are animals currently monitored? Conventionally, man-
agement and conservation of animal species are based on data
collection carried out by human
fi
eld workers who count animals,
observe their behavior, and/or patrol natural reserves. Such efforts
are time-consuming, labor-intensive, and expensive
3
. They can
also result in biased datasets due to challenges in controlling for
observer subjectivity and assuring high inter-observer reliability,
and often unavoidable responses of animals to observer
presence
4
,
5
. Human presence in the
fi
eld also poses risks to
wildlife
6
,
7
, their habitats
8
, and humans themselves: as an example,
many wildlife and conservation operations are performed from
aircraft and plane crashes are the primary cause of mortality for
wildlife biologists
9
. Finally, the physical and cognitive limitations
of humans unavoidably constrain the number of individual ani-
mals that can be observed simultaneously, the temporal resolu-
tion and complexity of data that can be collected, and the extent
of physical area that can be effectively monitored
10
,
11
.
These limitations considerably hamper our understanding of
geographic ranges, population densities, and community diversity
globally, as well as our ability to assess the consequences of their
decline. For example, humans conducting counts of seabird
colonies
12
and bats emerging from cave roosts
13
tend to sig-
ni
fi
cantly underestimate the number of individuals present.
Furthermore, population estimates based on extrapolation from a
small number of point counts have large uncertainties and can fail
to capture the spatiotemporal variation in ecological relation-
ships, resulting in erroneous predictions or extrapolations
14
.
Failure to monitor animal populations impedes rapid and effec-
tive management actions
3
. For example, insuf
fi
cient monitoring,
due in part to the dif
fi
culty and cost of collecting the necessary
data, has been identi
fi
ed as a major challenge in evaluating the
impact of primate conservation actions
15
and can lead to the
continuation of practices that are harmful to endangered
species
16
. Similarly, poaching prevention requires intensive
monitoring of vast protected areas, a major challenge with
existing technology. Protected area managers invest heavily in
illegal intrusion prevention and the detection of poachers. Despite
this, rangers often arrive too late to prevent wildlife crime from
occurring
17
. In short, while a rich tradition of human-based data
collection provides the basis for much of our understanding of
where species are found, how they live, and why they interact,
modern challenges in wildlife ecology and conservation are
highlighting the limitations of these methods.
Recent advances in sensor technologies are drastically
increasing data collection capacity by reducing costs and
expanding coverage relative to conventional methods (see the
section
New sensors expand available data types for animal
ecology
, below), thereby opening new avenues for ecological
studies at scale (Fig.
1
)
18
. Many previously inaccessible areas of
conservation interest can now be studied through the use of high-
resolution remote sensing
19
, and large amounts of data are being
collected non-invasively by digital devices such as camera traps
20
,
consumer cameras
21
, and acoustic sensors
22
. New on-animal bio-
loggers, including miniaturized tracking tags
23
,
24
and sensor
arrays featuring accelerometers, audiologgers, cameras, and other
monitoring devices document the movement and behavior of
animals in unprecedented detail
25
, enabling researchers to track
individuals across hemispheres and over their entire lifetimes at
high temporal resolution and thereby revolutionizing the study of
animal movement (Fig.
1
c) and migrations.
There is a mismatch between the ever-growing volume of raw
measures (videos, images, audio recordings) acquired for ecological
studies and our ability to process and analyze this multi-source
data to derive conclusive ecological insights rapidly and at scale.
Effectively, ecology has entered the age of big data and is
increasingly reliant on sensors, advanced methodologies, and
computational resources
26
. Central challenges to ef
fi
cient data
analysis are the sheer volume of data generated by modern col-
lection methods and the heterogeneous nature of many ecological
datasets, which preclude the use of simple automated analysis
techniques
26
. Crowdsourcing platforms like eMammal
(
emammal.si.edu
), Agouti (
agouti.eu
), and Zooniverse
(
www.zooniverse.org
) function as collaborative portals to collect
data from different projects and provide tools to volunteers to
annotate images, e.g., with species labels of the individuals therein.
Such platforms drastically reduce the cost of data processing (e.g.,
ref.
27
reports a reduction of seventy thousand dollars), but the
rapid increase in the volume and velocity of data collection is
making such approaches unsustainable. For example, in August
2021 the platform Agouti hosted 31 million images, of which only
1.5 million were annotated. This is mostly due to the manual
nature of the current annotation tool, which requires human
review of every image. In other words, methods for automatic
cataloging, searching, and converting data into relevant informa-
tion are urgently needed and have the potential to broaden and
enhance animal ecology and wildlife conservation in scale and
accuracy, address prevalent challenges, and pave the way forward
towards new, integrated research directives.
Machine learning (ML, see glossary in Supplementary Table 1)
deals with learning patterns from data
28
. Presented with large
quantities of inputs (e.g., images) and corresponding expected
outcomes, or labels (e.g., the species depicted in each image), a
supervised ML algorithm learns a mathematical function leading
to the correct outcome prediction when confronted with new,
unseen inputs. When the expected outcomes are absent, the (this
time unsupervised) ML algorithm will use solely the inputs to
extract groups of data points corresponding to typical patterns in
the data. ML has emerged as a promising means of connecting the
dots between big data and actionable ecological insights
29
and is
an increasingly popular approach in ecology
30
,
31
. A signi
fi
cant
share of this success can be attributed to deep learning (DL
32
), a
family of highly versatile ML models based on arti
fi
cial neural
networks that have shown superior performance across the
majority of ML use cases (see Table
1
and Supplementary
Table 2). Signi
fi
cant error reduction of ML and DL with respect to
traditional generalized regression models has been reported rou-
tinely for species richness and diversity estimation
33
,
34
. Likewise,
detection and counting pipelines moved from rough rule of thumb
extrapolations from visual counts in national parks to ML-based
methods with high detection rates. Initially, these methods pro-
posed many false positives which required further human
review
35
, but recent methods have been shown to maintain high
detection rates with signi
fi
cantly fewer false positives
36
.Asan
example, large mammal detection in the Kuzikus reserve in 2014
was improved signi
fi
cantly by improving the detection meth-
odologies, from a recall rate of 20%
35
to 80%
37
(for a common
75% precision rate). Finally, studies involving human operators
demonstrated that ML enabled massive speedups in complex tasks
such as individual and species recognition
38
,
39
and large-scale
tasks such as animal detection in drone surveys
40
. Recent advances
in ML methodology could accelerate and enhance various stages
of the traditional ecological research pipeline (see Fig.
2
), from
targeted data acquisition to image retrieval and semi-automated
population surveys. As an example, the initiative Wildlife
Insights
41
is now processing millions of camera trap images
automatically (17 million in August 2021), providing wildlife
conservation scientists and practitioners with the data necessary to
study animal abundances, diversity, and behavior. Besides pure
acceleration, use of ML also massively reduces analysis costs, with
reduction factors estimated between 2 and 10
42
.
A growing body of literature promotes the use of ML in various
ecological sub
fi
elds by educating domain experts about ML
PERSPECTIVE
NATURE COMMUNICATIONS | https://doi.org/10.1038/s41467-022-27980-y
2
NATURE COMMUNICATIONS
| (2022) 13:792 | https://doi.org/10.1038/s41467-022-27980-y | www.nature.com/naturecommunications
approaches
29
,
43
,
44
, their utility in capitalizing on big data
26
,
45
,
and, more recently, their potential for ecological inference (e.g.,
understanding the processes underlying ecological patterns,
rather than only predicting the patterns themselves)
46
,
47
. Clearly,
there is a growing interest in applying ML approaches to pro-
blems in animal ecology and conservation. We believe that the
challenging nature of ecological data, compounded by the size of
the datasets generated by novel sensors and the ever-increasing
complexity of state-of-the-art ML methods, favor a collaborative
approach that harnesses the expertise of both the ML and animal
ecology communities, rather than an application of off-the-shelf
ML methodologies to ecological challenges. Hence, the relation
between ecology and ML should not be unidirectional: integrating
ecological domain knowledge into ML methods is essential to
designing models that are accurate in the way they describe
animal life. As demonstrated by the rising
fi
eld of hybrid envir-
onmental algorithms (leveraging both DL and bio-physical
models
48
,
49
) and, more broadly, by theory-guided data
science
50
, such hybrid models tend to be less data-intensive, avoid
incoherent predictions, and are generally more interpretable than
Fig. 1 Examples of research acceleration by machine learning-based systems in animal ecology. a
The BirdNET algorithm
61
was used to detect Carolina
wren vocalizations in more than 35,000 h of passive acoustic monitoring data from Ithaca, New York, allowing researchers to document the gradual
recovery of the population following a harsh winter season in 2015.
b
Machine-learning algorithms were used to analyze movement of savannah herbivores
fi
tted with bio-logging devices in order to identify human threats. The method can localize human intruders to within 500 m, suggesting `sentinel anima
ls'
may be a useful tool in the
fi
ght against wildlife poaching
148
.
c
TRex, a new image-based tracking software, can track the movement and posture of hundreds
of individually-recognized animals in real-time. Here the software has been used to visualize the formation of trails in a termite colony
149
.
d
,
e
Pose
estimation software, such as DeepPoseKit (
d
)
75
and DeepLabCut (
e
)
74
,
142
allows researchers to track the body position of individual animals from video
imagery, including drone footage, and estimate 3D postures in the wild. Panels
b
,
c
, and
d
are reproduced under
CC BY 4.0
licenses. Panels
b
and
d
are
cropped versions of the originals; the legend for panel
b
has been rewritten and reorganized. Panel
e
is reproduced with permission from Joska et al.
142
.
NATURE COMMUNICATIONS | https://doi.org/10.1038/s41467-022-27980-y
PERSPECTIVE
NATURE COMMUNICATIONS
| (2022) 13:792 | https://doi.org/10.1038/s41467-022-27980-y | www.nature.com/naturecommunications
3
purely data-driven models. To reach this goal of an integrated
science of ecology and ML, both communities need to work
together to develop specialized datasets, tools, and knowledge.
With this objective in mind, we review recent efforts at the
interface of the two disciplines, present success stories of such
symbiosis in animal ecology and wildlife conservation, and sketch
an agenda for the future of the
fi
eld.
New sensors expand available data types for animal ecology
Sensor data provide a variety of perspectives to observe wildlife,
monitor populations, and understand behavior. They allow the
fi
eld to scale studies in space, time, and across the taxonomic tree
and, thanks to open science projects (Table
2
), to share data
across parks, geographies, and the globe
51
. Sensors generate
diverse data types, including imagery, soundscapes, and posi-
tional data (Fig.
3
). They can be mobile or static, and can be
deployed to collect information on individuals or species of
interest (e.g., bio-loggers, drones), monitor activity in a particular
location (e.g., camera traps and acoustic sensors), or document
changes in habitats or landscapes over time (satellites, drones).
Finally, they can also be opportunistic, as in the case of
community science. Below, we discuss the different categories of
sensors and the opportunities they open for ML-based wildlife
research.
Stationary sensors
. Stationary sensors provide close-range
continuous monitoring over long time scales. Sensors can be
image-based (e.g., camera traps) or signal-based (e.g., sound
recorders). Their high level of temporal resolution allows for
detailed analysis, including presence/absence, individual identi-
fi
cation, behavior analysis, and predator-prey interaction.
However, because of their stationary nature, their data is highly
spatiotemporally correlated. Based on where and when in the
world the sensor is placed, there is a limited number of species
that can be captured. Furthermore, many animals are highly
habitual and territorial, leading to very strong correlations
between data taken days or even weeks apart from a single
sensor
52
.
Camera traps
are among the most used sensors in recent
ML-based animal ecology papers, with more than a million
cameras already used to monitor biodiversity worldwide
20
.
Camera traps are inexpensive, easy to install, and provide
Table 1 Resources for machine and deep learning-based wildlife conservation.
Name
Description
URL
AIDE
150
Tasks: Annotation; detection; classi
fi
cation; segmentation
Free, open source, web-based, collaborative labeling platform speci
fi
cally designed for large-scale ecological image
analyses. Users can concurrently annotate up to billions of images with labels, points, bounding boxes, or pixel-wise
segmentation masks. AIDE tightly integrates ML models through Active Learning
151
, where annotators are asked to
provide inputs where the model is the least con
fi
dent. AIDE further offers functionality to share and exchange trained ML
models with other users of the system for collaborative annotation efforts in image campaigns across the globe.
GitHub
MegaDetector
36
Tasks: Detection
Free and open source detector based on deep learning hosted by Microsoft AI4Earth. The current model is trained with
the TensorFlow Object Detection API using several hundred thousand camera trap images labeled with bounding boxes
from a variety of ecosystems. The model identi
fi
es animals (not species-speci
fi
c), humans, and vehicles, and is robust to
novel sensor deployment locations and taxa not seen during training. Updates of the model, trained with additional data,
are periodically released. Microsoft AI4Earth provides support to assist ecologists in using the model, including a public
API for batch inference, and integration with commonly-used camera trap data management platforms such as TimeLapse
and Camelot.
GitHub
Wildbook
99
Tasks: Individual re-identi
fi
cation
Wildbook blends structured wildlife research with arti
fi
cial intelligence, community science, and computer vision to speed
population analysis and develop new insights to help
fi
ght extinction. They host community-run individual re-identi
fi
cation
systems and global data repositories for a broad and expanding set of species, including Grevy
s zebra, whale sharks,
manta rays, and many more.
URL
Wildlife Insights
41
Tasks: Filtering
Large-scale platform for camera trap data management with computer vision in the backend. Currently open for
whitelisted users, extensible via a waitlist. Wildlife Insights
fi
lters blank images and provides species identi
fi
cation for
images that the computer vision model scores highly, allowing expert ecologists to focus on labeling only challenging
images.
URL
DeepLabCut
74
Tasks: Pose estimation and behavioral analysis
Free and open-source pose estimation toolbox based on deep learning. Pre-trained models (for instance for primate faces
and bodies, as well as quadruped) as well as a light-weight, real-time version are available.
GitHub
DeepPoseKit
75
Tasks: Pose estimation and behavioral analysis
Free and open-source pose estimation toolbox based on deep learning.
GitHub
Fig. 2 Incorporating ML into the ecological scienti
fi
c process.
Traditional ecological research pipeline (colored text and boxes) and contributions of ML to
the different stages discussed in this paper (black text).
PERSPECTIVE
NATURE COMMUNICATIONS | https://doi.org/10.1038/s41467-022-27980-y
4
NATURE COMMUNICATIONS
| (2022) 13:792 | https://doi.org/10.1038/s41467-022-27980-y | www.nature.com/naturecommunications
high-resolution image sequences of the animals that trigger
them, suf
fi
cient to specify the species, sex, age, health,
behavior, and predator-prey interactions. Coupled with
population models, camera-trap data has also been used to
estimate species occurrence, richness, distribution, and
density
20
. But the popularity of camera traps also creates
challenges relative to the quantity of images and the need for
manual annotation of the collections: software tools easing
the annotation process are appearing (see, e.g., AIDE in
Table
1
) and many ecologists have already incorporated
open-source ML approaches for
fi
ltering out blank images
(such as the Microsoft AI4Earth MegaDetector
36
,see
Table
1
) into their camera trap work
fl
ows
52
54
.However,
problems related to lack of generality across geographies,
day/night acquisition, or sensors are still major obstacles to
production-ready accurate systems
55
. The increased scale of
available data due to de-siloing efforts from organizations
like Wildlife Insights (
www.wildlifeinsights.org
)and
LILA.science (
www.lila.science
) will help increase ML
accuracy and robustness across regions and taxa.
Bioacoustic sensors
are an alternative to image-based
systems, using microphones and hydrophones to study
vocal animals and their habitats
56
. Networks of static
bioacoustic sensors, used for passive acoustic monitoring
(PAM), are increasingly applied to address conservation
issues in terrestrial
57
, aquatic
58
, and marine
59
ecosystems.
Compared to camera traps, PAM is mostly unaffected by
light and weather conditions (some factors like wind still
play a role), senses the environment omnidirectionally, and
tends to be cost-effective when data needs to be collected at
large spatiotemporal scales with high resolution
60
. While
ML has been extensively applied to camera trap images, its
application to long-term PAM datasets is still in its infancy
and the
fi
rst DL-based studies are only starting to appear
(see Fig.
1
a, ref.
61
). Signi
fi
cant challenges remain when
utilizing PAM. First and foremost among these challenges
is the size of data acquired. Given the often continuous and
high-frequency acquisition rates, datasets often exceed the
terabyte scale. Handling and analyzing these datasets
ef
fi
ciently requires access to advanced computing infra-
structure and solutions. Second, the inherent complexity of
soundscapes requires noise-robust algorithms that general-
ize well and can separate and identify many animal sounds
of interest from confounding natural and anthropogenic
signals in a wide variety of acoustic environments
62
. The
third challenge is the lack of large and diverse labeled
datasets. As for camera trap images, species- or region-
speci
fi
c characteristics (e.g., regional dialects
63
) affect
algorithm performance. Robust, large-scale datasets have
begun to be curated for some animal groups (e.g.,
www.macaulaylibrary.org
and
www.xeno-canto.org
for
birds), but for many animal groups as well as relevant
biological and non-biological confounding signals, such
data is still nonexistent.
Table 2 Examples of community science projects in digital wildlife conservation.
Name
Spatial coverage
Sensor
Task
Ref.
iNaturalist
Global
Human photographers
Classi
fi
cation detection
132
SAVMAP
Kuzikus reserve, Namibia
UAV images
Detection
152
Zooniverse
Global
Images, text, video
Classi
fi
cation detection
153
iRecord
United Kingdom
Photographic records
Classi
fi
cation
154
Great Grevy
s Rally
Northern Kenya
Safari pictures
Classi
fi
cation detection identi
fi
cation
92
Fig. 3 A variety of sensors used in animal ecology.
Studies frequently combine data from multiple sensors at the same geographic location, or data from
multiple locations to achieve deeper ecological insights. Sentinel-2 (ESA) satellite image courtesy of the U.S. Geological Survey.
NATURE COMMUNICATIONS | https://doi.org/10.1038/s41467-022-27980-y
PERSPECTIVE
NATURE COMMUNICATIONS
| (2022) 13:792 | https://doi.org/10.1038/s41467-022-27980-y | www.nature.com/naturecommunications
5