Detailed audio-video joint captioning is essential for multimodal video understanding and generation. However, prior works are constrained by three main limitations: (1) the scarcity of high-quality public datasets with fine-grained audio-visual joint captions; (2) reinforcement-learning methods that rely on coarse rew...
Ming-Yang Wu, Kai-Tuo Feng, Bohao Li et al.· 0 citations
JarvisHub is introduced, a canvas-native creative agent harness for long-horizon multimodal creation, where agents can progressively plan, generate, revise, and organize multimodal projects while users remain able to inspect, guide, and intervene throughout the process.
Twin, a unified continuous token space formed by channel-wise concatenating ViT and VAE features on the same token grid, so the sequence length is unchanged and attention cost does not increase is proposed, which performs competitively on multimodal understanding benchmarks and improves reconstruction fidelity.
Kaixiong Gong, Xin Cai, Bin Lin et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
