PACT: A Pragmatic Prompting Framework for Expressive and Voice-Preserving Speech-to-Speech Translation
TL;DR
PACT is proposed, a target-space pragmatic prompting framework for expressive and voice-preserving S2ST, which achieves the best overall performance among evaluated systems across content fidelity, voice preservation, and expression preservation metrics.
Abstract
Speech-to-speech translation should preserve not only what is said, but also who says it and how it is said. Existing end-to-end S2ST, speech MLLM, and cascade systems can preserve content or timbre, but often struggle to realize emotion, tone, rhythm, pauses, and pragmatic attitude in the target language. We propose PACT, a target-space pragmatic prompting framework for expressive and voice-preserving S2ST. PACT extracts source pragmatic cues, projects them into the target-language expression space, calibrates the projected representation to a realizable target subspace, and converts it into continuous prompts for a frozen speech generator. On bidirectional Spanish-English TOPI evaluation, PACT achieves the best overall performance among evaluated systems across content fidelity, voice preservation, and expression preservation metrics.
