Detailed audio-video joint captioning is essential for multimodal video understanding and generation. However, prior works are constrained by three main limitations: (1) the scarcity of high-quality public datasets with fine-grained audio-visual joint captions; (2) reinforcement-learning methods that rely on coarse rew...
Ming-Yang Wu, Kai-Tuo Feng, Bohao Li et al.· 0 citations
The first capability-driven benchmark designed to evaluate proactive agents in dynamic, real-world settings is introduced, and comprehensive comparisons across both models and frameworks show how base model capabilities and agent framework designs jointly shape performance in real-world environments.
Zhekai Chen, Chengqi Duan, Kaiyue Sun et al.· arXiv.org· 1 citation
Twin, a unified continuous token space formed by channel-wise concatenating ViT and VAE features on the same token grid, so the sequence length is unchanged and attention cost does not increase is proposed, which performs competitively on multimodal understanding benchmarks and improves reconstruction fidelity.
Kaixiong Gong, Xin Cai, Bin Lin et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.