Soft Prompt Tuning for Multilingual Natural Language Understanding in Human–Robot Interaction with Bidirectional Cross-Attention

Abstract

Natural language understanding (NLU) enables intuitive human–robot interaction (HRI). It requires joint modeling of intent classification and named entity recognition (NER). Existing approaches suffer from three key limitations. Fine-tuning large multilingual pretrained models on domain-specific HRI data incurs high computational overhead. Intent classification and NER are typically treated as independent tasks, despite their semantic interdependence. Labeled HRI datasets for Indian languages are absent, to the best of our knowledge, limiting system accessibility for native-speaking communities. This paper proposes SoftPromptIndicBERTCRF, a parameter-efficient multilingual NLU architecture built on a frozen, 33.4 M-parameter IndicBERT backbone. The architecture prepends 50 learnable soft prompt tokens, initialized from task-specific HRI vocabulary, to the input sequence. A bidirectional cross-attention module, with 8 attention heads, models the mutual dependency between intent and NER. A Conditional Random Field (CRF) decoder enforces valid Begin–Inside–Outside (BIO) tag sequences during entity decoding. Custom HRI datasets were constructed for English, Hindi, and Malayalam, covering 13 intent classes and 7 entity types. The proposed framework provides 99.35% accuracy for intent detection and a 98.77% F1 score for NER in all three languages. The latency for end-to-end pipelining involving ASR transcriptions and NLU inferences is 0.78 to 0.92 s on average in all three languages, thereby validating its utility for real-time application in robots.