Proceedings of International Conference on Applied Innovation in IT  ·  2026/07/22  ·  Vol. 14  ·  Issue 4  ·  pp. 1171–1177
Methods and Approaches for Mathematical Optimization of Hyperparameters of Machine Learning Models
Fakhriddin Shodiev, Sharipboy Fayzullaev, Yulduz Nematjonova, Shaun Billiones, Shafoat Eshonkhulova, Shakhnoza Samandarova, Laylo Ernazarova and Valida Chorieva
This paper investigates the mathematical optimization of machine learning hyperparameters in the context of early chronic kidney disease (CKD) detection. This study compares two high-performing ensemble classifiers - Random Forest and XGBoost - under five tuning strategies: default scikit-learn settings, Grid Search, Random Search, a Genetic Algorithm (GA), and Ant Colony Optimization (ACO). Logistic regression, k-nearest neighbors, support vector machines, decision trees, and Naive Bayes were evaluated as baseline comparators under default settings only; their tuning results are reported in the supplementary material. The main experimental comparison focuses on RF and XGBoost, for which full five-strategy benchmarks, sensitivity analysis, and runtime profiling are available. Grid Search and Random Search are shown to be impractical at scale: for Random Forest, Grid Search evaluated 432 configurations in 4.2 hours and Random Search evaluated 200 in 1.8 hours, neither reliably reaching high-sensitivity solutions. GA-optimized Random Forest and XGBoost reached AUC-ROC values of 0.963 and 0.968, respectively, while ACO-optimized variants achieved 0.969 and 0.975. Critically, sensitivity for the CKD class the clinically most important metric rose from 0.864 to 0.924 (ACO) for Random Forest. Paired Wilcoxon signed-rank tests confirmed that all GA/ACO improvements over Grid Search are statistically significant (p < 0.01). The results establish that evolutionary optimization offers a practically favorable accuracy-to-runtime trade-off for medical classification tasks where the cost of false negatives is high.
Hyperparameter Optimization Genetic Algorithm Ant Colony Optimization Ensemble Models Random Forest XGBoost Chronic Renal Illness Medical Diagnostics
References
  1. M. M. Talipov, “Computational modeling and analysis of mechanical power consumption in train assemblers’ work,” in Proc. Int. Conf. on Applied Innovations in IT (ICAIIT), vol. 13, no. 2, pp. 419-426, 2025, [Online]. Available: https://doi.org/10.25673/120513.
  2. M. Shukurova, K. Ruziev, M. Talipov, and G. Talipova, “3D modeling of filtration in oil and gas reservoirs based on satellite data,” Proceedings of International Conference on Applied Innovation in IT, vol. 14, no. 1, pp. 267-271, Mar. 2026, [Online]. Available: https://doi.org/10.25673/123569.
  3. M. Shukurova, M. Talipov, K. Ruziev, and K. Jurayeva, “Advanced geospatial monitoring of oil and gas infrastructure via satellite data,” Mathematical Models in Engineering, vol. 12, no. 2, pp. 190-201, Jun. 2026, [Online]. Available: https://doi.org/10.21595/mme.2026.25328.
  4. A. Mamatov, X. Sotvoldiyev, M. Talipov, S. Shaumarov, K. Gafarbayli, and D. Bekmirzaev, “Metrological calibration and uncertainty evaluation of MEMS accelerometers for structural health monitoring in seismic regions,” Vibroengineering Procedia, vol. 62, pp. 140-144, Jun. 2026, [Online]. Available: https://doi.org/10.21595/vp.2026.26360.
  5. K. Laxmikant, R. Bhuvaneswari, and B. Natarajan, “An Efficient Approach to Detect Diabetes using XGBoost Classifier,” in 2023 Winter Summit on Smart Computing and Networks (WiSSCoN), Chennai, India, 2023, pp. 1-8, [Online]. Available: https://doi.org/10.1109/WiSSCoN56857.2023.10133854.
  6. S. A. Tiruneh, T. T. T. Vu, D. L. Rolnik, H. J. Teede, and J. Enticott, “Machine Learning Algorithms Versus Classical Regression Models in Pre-Eclampsia Prediction: A Systematic Review,” Current Hypertension Reports, vol. 26, no. 7, pp. 309-323, Jul. 2024, [Online]. Available: https://doi.org/10.1007/s11906-024-01297-1.
  7. R. Hephzibah, A. H. Christinal, R. Jayanthi, D. A. Chandy, and C. Bajaj, “A Novel Ensemble Classifier Framework for Accurate Fetal Heart Rate Classification,” in 2023 4th International Conference on Signal Processing and Communication (ICSPC), Coimbatore, India, 2023, pp. 321-324, [Online]. Available: https://doi.org/10.1109/ICSPC57692.2023.10125713.
  8. N. Fayzullo, S. Sariyev, and Y. Sherzodjon, “Analyzing the Effectiveness of Ensemble Methods in Solving Multi-Class Classification Problems,” in 2025 International Russian Smart Industry Conference (SmartIndustryCon), Sochi, Russian Federation, 2025, pp. 788-793, [Online]. Available: https://doi.org/10.1109/SmartIndustryCon65166.2025.10986248.
  9. S. Pawar, P. Bagal, P. Shukla, and A. Dawkhar, “Detection of Breast Cancer using Machine Learning Classifier,” in 2021 Asian Conference on Innovation in Technology (ASIANCON), Pune, India, 2021, pp. 1-5, [Online]. Available: https://doi.org/10.1109/ASIANCON51346.2021.9544767.
  10. T. Chen and C. Guestrin, “XGBoost: a scalable tree boosting system,” in Proc. 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), New York, NY, USA, 2016, pp. 785-794.
  11. L. Gosink, et al., “Characterizing and Visualizing Predictive Uncertainty in Numerical Ensembles Through Bayesian Model Averaging,” IEEE Transactions on Visualization and Computer Graphics, vol. 19, no. 12, pp. 2703-2712, Dec. 2013, [Online]. Available: https://doi.org/10.1109/TVCG.2013.138.
  12. J. H. Friedman, “Greedy Function Approximation: A Gradient Boosting Machine,” Annals of Statistics, vol. 29, pp. 1189-1232, 2001, [Online]. Available: https://doi.org/10.1214/aos/1013203451.
  13. H. Alibrahim and S. A. Ludwig, “Hyperparameter Optimization: Comparing Genetic Algorithm against Grid Search and Bayesian Optimization,” in 2021 IEEE Congress on Evolutionary Computation (CEC), Kraków, Poland, 2021, pp. 1551-1559, [Online]. Available: https://doi.org/10.1109/CEC45853.2021.9504761.
  14. S. Sariyev, G. Negmatova, and A. Sayidkulov, “Preparation Datasets Based on Artificial Intelligence Models and Classification Algorithms,” in 2025 International Russian Automation Conference (RusAutoCon), Sochi, Russian Federation, 2025, pp. 260-266, [Online]. Available: https://doi.org/10.1109/RusAutoCon65989.2025.11177339.
  15. S. Akhatkulov, I. Yalgoshev, and Z. Urinboyev, “Vehicle CO2 Emission Prediction Using Deep Learning and Ensemble Machine Learning Methods,” in 2025 International Russian Automation Conference (RusAutoCon), Sochi, Russian Federation, 2025, pp. 819-824, [Online]. Available: https://doi.org/10.1109/RusAutoCon65989.2025.11177377.
  16. M. Dorigo, M. Birattari, and T. Stutzle, “Ant colony optimization,” IEEE Computational Intelligence Magazine, vol. 1, no. 4, pp. 28-39, Nov. 2006, [Online]. Available: https://doi.org/10.1109/MCI.2006.329691.
  17. X. Xu, J. Peng, W. Zhao, W. Nai, Z. Yang, and Y. Xing, “Lasso Regression Based on t-Distribution Coot Optimization Algorithm,” in 2022 IEEE 6th Information Technology and Mechatronics Engineering Conference (ITOEC), Chongqing, China, 2022, pp. 1920-1924, [Online]. Available: https://doi.org/10.1109/ITOEC53115.2022.9734538.
  18. S. Sariyev, I. Yalgoshev, and M. Nuriddinova, “Traditional Methods and Modern Approaches Based on Ensemble Algorithms for Decision-Making in Diagnostics Using Medical Data,” in 2025 International Russian Automation Conference (RusAutoCon), Sochi, Russian Federation, 2025, pp. 971-975, [Online]. Available: https://doi.org/10.1109/RusAutoCon65989.2025.11177423.
  19. A. Taufiq, S. Yulianti, A. Rahmatulloh, I. Darmawan, and R. Rizal, “Comparison of Hyperparameter Tuning Techniques on KNN Algorithm to find the Best K Value using Grid Search and Random Search Methods,” in 2024 7th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), Yogyakarta, Indonesia, 2024, pp. 180-186, [Online]. Available: https://doi.org/10.1109/ISRITI64779.2024.10963519.
  20. C. Di Francescomarino, et al., “Genetic algorithms for hyperparameter optimization in predictive business process monitoring,” Information Systems, vol. 74, pp. 67-83, 2018, [Online]. Available: https://doi.org/10.1016/j.is.2018.01.003.
  21. E. Zarate-Perez and R. Sebastián, “Optimizing the sizing of residential microgrids using a genetic algorithm as a decision support model,” Management of Environmental Quality, vol. 37, no. 2, pp. 336-357, 2025, [Online]. Available: https://doi.org/10.1108/MEQ-01-2025-0043.
  22. A. S. Wicaksono and A. A. Supianto, “Hyper Parameter Optimization Using Genetic Algorithm on Machine Learning Methods for Online News Popularity Prediction,” International Journal of Advanced Computer Science and Applications (IJACSA), vol. 9, no. 12, pp. 263-267, 2018, [Online]. Available: https://doi.org/10.14569/IJACSA.2018.091238.
  23. E. Elgeldawi, A. Sayed, A. R. Galal, and A. M. Zaki, “Hyperparameter Tuning for Machine Learning Algorithms Used for Arabic Sentiment Analysis,” Informatics, vol. 8, no. 4, Art. no. 79, Nov. 2021, [Online]. Available: https://doi.org/10.3390/informatics8040079.
  24. S. Akhatkulov and I. Yalgoshev, “Air quality prediction based on machine learning techniques,” AIP Conference Proceedings, vol. 3356, no. 1, Art. no. 030005, Sep. 2025, [Online]. Available: https://doi.org/10.1063/5.0296121.


Proceedings of the International Conference on Applied Innovations in IT by Anhalt University of Applied Sciences is licensed under CC BY-SA 4.0
 ·  This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

ICAIIT 2026
International Conference on Applied Innovation in IT
Navigation
Publisher
ISSN2199-8876
Location Anhalt University of Applied Sciences
Phone +49 (0) 3496 67 5611
Address Building 01, Room 425
Bernburger Str. 55
D-06366 Köthen, Germany
Open Access License

All works are licensed under the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0), unless otherwise noted.

Published by ICAIIT in cooperation with Anhalt University of Applied Sciences.

© 2026 ICAIIT — International Conference on Applied Innovations in IT. Anhalt University of Applied Sciences, Köthen, Germany.
Visitors: site traffic counter