A Hybrid Spiking-Attention Transformer Model for Robust and Efficient Speech Emotion Recognition on Multi-Dataset Benchmarks

Samah Abbas Ali and Jamal Mustafa Abbas

References:

1. C. Parlak, “Cochleogram-Based Speech Emotion Recognition with the Cascade of Asymmetric Resonators with Fast-Acting Compression Using Time-Distributed Convolutional Long Short-Term Memory and Support Vector Machines,” Biomimetics, vol. 10, no. 3, p. 167, 2025, doi: 10.3390/biomimetics10030167.
2. D. Y. Badawood and F. M. Aldosari, “Enhanced Deep Learning Techniques for Real-Time Speech Emotion Recognition in Multilingual Contexts,” Engineering, Technology & Applied Science Research, vol. 14, no. 6, pp. 18662–18669, 2024.
3. S. Yaser, M. S. I. Sadhin, and R. H. Ifty, “Speech Emotion Recognition using Transfer Learning Approach and Real-Time Evaluation in English and Bengali Language,” unpublished.
4. “Speech Emotion Recognition (en).” [Online]. Available: https://www.kaggle.com/datasets/dmitrybabko/speech-emotion-recognition-en
5. , accessed Jan. 19, 2025.
6. M. A. Uddin, M. S. U. Chowdury, M. U. Khandaker, N. Tamam, and A. Sulieman, “The efficacy of deep learning-based mixed model for speech emotion recognition,” Computer Materials & Continua, vol. 74, no. 1, pp. 1709–1722, 2022.
7. C. Tan, M. Šarlija, and N. Kasabov, “NeuroSense: Short-term emotion recognition and understanding based on spiking neural network modelling of spatio-temporal EEG patterns,” Neurocomputing, vol. 434, pp. 137–148, 2021.
8. M. Ezz-Eldin, A. A. M. Khalaf, H. F. A. Hamed, and A. I. Hussein, “Efficient feature-aware hybrid model of deep learning architectures for speech emotion recognition,” IEEE Access, vol. 9, pp. 19999–20011, 2021.
9. W. Alzhrani, M. Doborjeh, Z. Doborjeh, and N. Kasabov, “Emotion recognition and understanding using EEG data in a brain-inspired spiking neural network architecture,” in Proc. Int. Joint Conf. Neural Networks (IJCNN), 2021, pp. 1–9.
10. B. Wang, J. Xu, L. Chen, Q. Zhang, and Y. Li, “Spiking Emotions: Dynamic Vision Emotion Recognition Using Spiking Neural Networks,” in Proc. AHPCAI, 2022, pp. 50–58.
11. K. Mountzouris, I. Perikos, and I. Hatzilygeroudis, “Speech emotion recognition using convolutional neural networks with attention mechanism,” Electronics, vol. 12, no. 20, p. 4376, 2023, doi: 10.3390/electronics12204376.
12. R. Ullah, Y. Zhang, S. Ali, H. Kim, and T. Lee, “Speech emotion recognition using convolution neural networks and multi-head convolutional transformer,” Sensors, vol. 23, no. 13, p. 6212, 2023, doi: 10.3390/s23136212.
13. C. S. A. Kumar, A. Das Maharana, S. M. Krishnan, S. S. S. Hanuma, G. J. Lal, and V. Ravi, “Speech emotion recognition using CNN-LSTM and vision transformer,” in Proc. Int. Conf. Innovations in Bio-Inspired Computing and Applications, 2022, pp. 86–97.
14. W. Li, C. Fang, Z. Zhu, C. Chen, and A. Song, “Fractal spiking neural network scheme for EEG-based emotion recognition,” IEEE J. Transl. Eng. Health Med., vol. 12, pp. 106–118, 2023.
15. T.-W. Kim and K.-C. Kwak, “Speech emotion recognition using deep learning transfer models and explainable techniques,” Applied Sciences, vol. 14, no. 4, p. 1553, 2024, doi: 10.3390/app14041553.
16. X. Tang, J. Huang, Y. Lin, T. Dang, and J. Cheng, “Speech emotion recognition via CNN-transformer and multidimensional attention mechanism,” Speech Communication, p. 103242, 2025, doi: 10.1016/j.specom.2025.103242.
17. Z. Wei, C. Ge, C. Su, R. Chen, and J. Sun, “A Deep Learning Model for Speech Emotion Recognition on RAVDESS Dataset,” Int. J. Adv. Comput. Sci. Appl., vol. 16, no. 5, 2025.