Deep learning to monitor the deep sea: a category‐oriented automated framework for quantifying animal behavior at hydrothermal vents

Abstract

Increasing evidences show that fauna living in deep‐sea ecosystems exhibit various complex behaviors that remain understudied. Although several in situ observatories have been set up around the globe, the use of their imagery data to gain insights into animal behavior across temporal scales remains constrained by the labor‐intensive nature of the analyses. Here, we developed an automated deep‐learning‐based pipeline with multi‐target tracking. Our pipeline integrates YOLOv8 object detection with category‐specific approaches to characterize spatial–temporal behavior patterns (i.e., mobility and trajectory). This pipeline was applied to video imagery from the Lucky Strike hydrothermal vent field, Mid‐Atlantic Ridge as a case study. We tailored tracking methods to three functional mobility groups: Sedentary, Simple Motile, and Complex Motile, represented by the mussel Bathymodiolus azoricus , the shrimp Mirocaris fortunata , and the crab Segonzacia mesatlantica , respectively. Tracking performance was evaluated using standardized multi‐object tracking metrics: the Hungarian algorithm showed strong manual agreement for Sedentary taxa ( r  = 0.792), BoTSORT excelled for Simple Motile (HOTA = 0.956, 0.012% identity switches), and DeepSORT achieved best HOTA = 0.727 for Complex Motile. Our results reveal: (1) daily displacement of the mussels, (2) a potential semi‐lunar rhythmic pattern in the movement of the shrimps (14.76‐day periodicity) modulated by the proximity to vent emissions and seasonal factors, and (3) acute stress responses in the crabs linked to localized thermal exposure to hot vent fluids. Our pipeline provides a scalable detection‐tracking‐behavior analysis framework, enabling statistical identification of infra‐monthly behavioral patterns that are difficult to quantify through manual annotations alone. As a result, it can be readily adapted to other deep‐sea ecosystems for standardized multi‐species behavioral monitoring.