Together, DeepLabSAM, MTMB dataset, and MouseLite provide a scalable framework for robust, cross-assay behavioural pose estimation of freely interacting mice, and substantially reduces misassigned keypoints under heavy occlusion.
Abstract
Behavioural experiments often hinge on moments when animals crowd together. In these situations, existing pose-estimation methods usually fail because detected crops include neighbouring animals, and their body parts are easily confused between individuals. We solved it with DeepLabSAM, a pipeline that segments and tracks every animal from a single text prompt using the SAM 3 foundation model, followed by a mask-gated pretrained pose estimation network from DeepLabCut. Across recordings of multiple interacting mice, the proposed method substantially reduces misassigned keypoints under heavy occlusion. Furthermore, we used DeepLabSAM to automatically construct the Multi-Task Mouse Behaviour (MTMB) dataset, comprising 14,300 frames spanning nine behavioural assays, with each instance annotated for bounding box, mask, identity, and pose. SuperAnimal pretrained pose models transferred unpredictably across this dataset. Conversely, MouseLite, our RF-DETR pose-estimation model trained on the MTMB dataset, matched their in-distribution accuracy, generalized better to unseen assays, and achieved real-time (faster-than-acquisition) inference. Combined, DeepLabSAM, MTMB dataset, and MouseLite provide a scalable framework for robust, cross-assay behavioural pose estimation of freely interacting mice.
Robust 6D object pose estimation is essential for enabling intelligent systems to interact with the physical world. While existing methods have achieved notable progress, they remain constrained by two major limitations: poor scalability when new objects are introduced and severe performance degradation caused by catas...
Long Tian, Yang Liu, Jun-Lin Fang et al.· IEEE Transactions on Image P...· 0 citations
Automatically estimating animal poses from videos is important for studying animal behavior. Existing methods do not perform reliably, partly because they rely on high-quality training data, and existing such datasets are small. It is challenging and expensive to create such data, due to the large variations in animal...
Zhuo-Yang Pan, Bo-Xiao Pan, Guan-Dao Yang et al.· International Journal of Com...· 1 citation
Across benchmark videos of 10 mice, 10 fish, 32 beetles, 32 ants with one cricket, 60 flies, and 368 ants, AMADEUS outperformed state-of-the-art methods in identity tracking and keypoint localization.
Yusuke Notomi, Kentarou Matsumura, S. Dobata· bioRxiv· 0 citations
3D human pose estimation from single-view images is a fundamental and challenging task in computer vision, the key of which lies in effectively capturing the complex spatial dependencies between human joint points. Existing methods usually rely on a single computational paradigm: Graph Convolutional Networks (GCN) exce...
Human pose estimation using inertial measurement units (IMUs) has gained increasing attention due to its ability to enable camera-free motion capture for numerous human-centered applications. However, estimating full-body pose from only one to three daily carried commercial-off-the-shelf IMU devices (e.g., smartphones,...
Han-Yu Zeng, Hui Ji, Peng-Fei Zhou· Proceedings of the 32nd Annu...· 0 citations
While deep neural networks have significantly advanced 2D Human Pose Estimation, their performance relies heavily on massive datasets with costly, dense skeletal annotations. In practical scenarios, while raw data is abundant, the high costs of manual annotation necessitate budget-constrained approaches. Active learnin...
Kadir Rabah, Julien Aligon, Xavier Baril et al.· Proceedings of the 35th ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.