Prior-enhanced cross-modal vibration and acoustic fusion network based on directed attention for machine fault diagnosis

Abstract

Existing intelligent fault diagnosis methods based on multimodal fusion face the problem of significant differences in the representation capabilities of different modal data for machine faults, making it difficult to achieve optimal cross-modal data fusion and accurate fault identification. This study proposes a prior-enhanced cross-modal vibration and acoustic data fusion network based on directed attention mechanisms to address the aforementioned issue. First, through preliminary experiments in fault diagnosis, the differences in fault sensitivity between vibration and acoustic data are measured to determine the prior dominant data modality. Then, based on the directed cross-attention mechanism, a prior-dominant modality-weighted fusion of vibration and acoustic data features is realized. This process allows for unidirectional feature information transfer from the dominant data modality to the weaker one, avoiding reverse information contamination. Thus, cross-modal fusion features that are more sensitive to machine faults can be extracted. Finally, the extracted cross-modal fusion features are used to achieve fault diagnosis. The results of two machine fault experiments demonstrate that, compared with the state-of-the-art methods, the proposed method can achieve a significant leading advantage in the same diagnostic tasks.