Malicious Traffic Detection in 5G Networks Using CNN and LSTM Models

Astrakhantsev, Andrii; Kapuler, Dmitriy; Globa, Inna

doi:Under indexing

Proceedings of International Conference on Applied Innovation in IT · 2026/04/22 · Vol. 14 · Issue 2 · pp. 37–42

Malicious Traffic Detection in 5G Networks Using CNN and LSTM Models

Andrii Astrakhantsev, Dmitriy Kapuler, Inna Butko and Larysa Globa

📄 Download PDF DOI: Under indexing

Abstract

Network security has become a critical concern with the exponential growth of cyber threats targeting modern communication infrastructures. With infrastructure upgrades and a significant increase in the number of information sources, the attack landscape has changed and new threats have emerged. However, the main and the most dangerous threats remain those associated with DoS and DDoS attacks. This work is devoted to identifying precisely these types of attacks. This research investigates the application of Convolutional Neural Networks (CNNs) for binary classification of network traffic to distinguish between benign and malicious activities. The study compares the performance of various neural network architectures, including traditional CNN, Dense, LSTM, and hybrid CNN-LSTM models, using network flow features extracted from traffic data. The experimental results demonstrate that the baseline CNN model achieves best performance with accuracy (0.8452) and precision (0.9998). Proposed approach can be implemented on the edge elements of network (like a baseband unit). It allows making an effective on-fly solution for network intrusion detection systems.

Keywords

Mobile Networks Traffic Classification Anomaly Detection Long Short-Term Memory (LSTM). CNN Accuracy Precision.

References

E. Egho-Promise, G. Asante, and H. Balisane, “Cybersecurity implications of 5G networks: Threats, potential vulnerabilities, and their implications for national security and privacy,” Indonesian Journal of Electrical Engineering and Informatics (IJEEI), vol. 13, no. 4, pp. 840-855, Dec. 2025, doi: 10.52549/ijeei.v13i4.6573.
H. Gang, H. Zhang, and Z. Zhang, “AI-based malicious encrypted traffic detection in 5G data collection and secure sharing,” Electronics, 2024.
Y. Pan, X. Zhang, and H. Jiang, “A network traffic classification method based on graph convolution and LSTM,” 2021, doi: 10.1109/ACCESS.2021.3128181.
R. Pell, M. Shojafar, D. Kosmanos, and S. Moschoyiannis, “Service classification of network traffic in 5G core networks using machine learning,” in Proc. IEEE International Conference on Edge Computing and Communications (EDGE), Chicago, IL, USA, pp. 309-318, 2023, doi: 10.1109/EDGE60047.2023.00053.
V. Pham, E. Seo, and T.-M. Chung, “Lightweight convolutional neural network based intrusion detection system,” Journal of Communications, vol. 15, no. 11, pp. 808-817, Nov. 2020, doi: 10.12720/jcm.15.11.808-817.
S. Al-Eidi, O. Darwish, G. Husari, Y. Chen, and M. Elkhodr, “Convolutional neural network structure to detect and localize CTC using image processing,” in Proc. IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS), Toronto, ON, Canada, pp. 1-7, 2022, doi: 10.1109/IEMTRONICS55184.2022.9795734.
B. Bousalem, V. Silva, R. Langar, and S. Cherrier, “DDoS attacks detection and mitigation in 5G and beyond networks: A deep learning-based approach,” in Proc. IEEE Global Communications Conference (GLOBECOM), Rio de Janeiro, Brazil, pp. 1259-1264, 2022, doi: 10.1109/GLOBECOM48099.2022.10001562.
Y. Huo, W. Liang, J. Chen, and S. Zhuang, “LightGuard: A lightweight malicious traffic detection method for Internet of Things,” IEEE Internet of Things Journal, vol. 11, pp. 28566-28577, 2024.
A. Ferriyan, A. H. Thamrin, K. Takeda, and J. Murai, “Encrypted malicious traffic detection based on Word2Vec,” Electronics, vol. 11, p. 679, 2022.
L. Globa, A. Astrakhantsev, and S. Tsukanov, “Classification of network traffic using machine learning methods,” Problemi telekomunìkacìj, no. 2, pp. 3-13, 2023, doi: 10.30837/pt.2023.2.01.
L. Globa, A. Astrakhantsev, and S. Tsukanov, “Comparison of the machine learning algorithms for traffic classification in 5G network,” in Proc. IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom), Tbilisi, Georgia, pp. 125-130, 2024, doi: 10.1109/BlackSeaCom61746.2024.10646279.
M. S. Khan, B. Farzaneh, N. Shahriar, and M. M. Hasan, “DoS/DDoS Attack Dataset of 5G Network Slicing,” IEEE Dataport, Sept. 25, 2023, doi: 10.21227/32k1-dr12.
F. Karim, S. Majumdar, and H. Darabi, “Multivariate LSTM-FCNs for time series classification,” Neural Networks, vol. 116, pp. 237-245, 2019, doi: 10.1016/j.neunet.2019.04.014.
Y. Wang, “Deep learning-based network intrusion detection systems,” Applied and Computational Engineering, vol. 109, no. 1, pp. 179-188, 2024, doi: 10.54254/2755-2721/2024.18104.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735-1780, Nov. 1997, doi: 10.1162/neco.1997.9.8.1735.
H. I. Fawaz, Deep Learning for Time Series Classification, Ph.D. dissertation, Université de Haute Alsace - Mulhouse, 2020, [Online]. Available: https://theses.hal.science/tel-03715016v1.