Local Prototype Reconstruction for Text-Compatible Speech-to-LLM Bridge Pretraining
Speech-to-LLM systems often connect a frozen speech encoder to a frozen large language model (LLM) through a small trainable bridge. The bridge is usually treated as plumbing, but it in fact defines the geometry of the speech-to-LLM interface, and the pretraining objective decides whether that interface provides a reus...
Xin-Nian Zhao, Chia-Hua Wu, Pu Wang et al.
· 0 citations