Video large language models (Vid-LLMs) excel at diverse video-language tasks by reasoning over selected frames. However, frame selection for long videos remains challenging, as it requires retrieving relevant frames distributed across segments from a large candidate pool given complex queries. This paper investigates d...
Yiyang Huang, Yi-Tian Zhang, Yi-Zhou Wang et al.· 0 citations
Sparsely activated Mixture-of-Experts (MoE) models increase model capacity without a proportional increase in per-token computation. Dense-to-MoE upcycling reuses pretrained dense models to construct such systems, commonly by copying feed-forward networks (FFNs) into independently trained experts. We introduce MASKerad...
Ming-Yuan Zhang, Yue Bai, Zhong-Ruo Wang et al.· 0 citations
This study examines a complementary perspective on LTH and proposes the Dual Lottery Ticket Hypothesis (DLTH), enabling general training for sparse networks and introduces Sharpness-Aware Random Sparse Network Transformation (SA-RST), which uses a regularization term to extrude information from weights slated for maski...
Yue Bai, Mingyuan Zhang, Huan Wang et al.· IEEE Transactions on Pattern...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.