Research Information System
IV. Uluslararası Sağlıkta Yapay Zekâ Kongresi, İzmir, Turkey, 15 - 17 November 2024, pp.1, (Summary Text)
Introduction-Aim: Childhood immunizations have a vital role in protecting public health and preventing the spread of infectious diseases. Childhood immunization protects children against diseases such as measles, diphtheria, whooping cough, and polio, which can lead to fatal and serious health problems, and enables individuals to step into a healthy life. Vaccines not only provide individual protection but also contribute to the control of infectious diseases in the community by supporting herd immunity. Societies with high vaccination rates prevent the spread of these diseases and reduce the risk of epidemics.
This study aims to conduct a systematic review to comprehensively examine ML models used in the prediction of childhood vaccination rates. The main objective of the study is to reveal the contexts in which different ML algorithms are most effective in predicting childhood immunization rates, to analyze the accuracy of these models in detail, and to contribute to the existing body of knowledge in this field. This analysis aims to determine which models are more appropriate and effective under certain populations, geographical regions, or socioeconomic conditions by comparing the success of various models.
Materials-Methods: This systematic review was conducted in accordance with the ‘Preferred Reporting Items for Systematic Reviews (PRISMA)’. The inclusion and exclusion criteria of the studies included in this study were determined according to the PICOS method. Inclusion criteria for this study: (1) Population: Children, (2) Intervention: Studies involving vaccine intervention, (3) Comparison: Studies using machine learning methods, (4) Outcomes: Prediction by machine learning methods, (5) Study design: Studies published in English between 2014 and 2024 that included machine learning methods of original artificial intelligence were included. In this systematic review, articles published in English between 2014 and 2024 were included. In the study, a literature search was performed in the ‘Web of Science, Google Scholar, Pubmed, and Scopus’ databases between 10-20 October 2024 using different combinations of the keywords ‘child’, ‘children’, ‘vaccine’, ‘vaccination’, ‘immunization', 'machine learning', ’rate’ and ‘prediction’. It was aimed to reach all studies related to the subject in the search of databases. Reference lists of included studies and previous systematic reviews were checked for additional searches. Three investigators independently performed the selection of studies. Initially, duplicate studies were excluded, and studies were selected if they fulfilled the search criteria when screened by title, abstract, and full text, respectively. The titles and abstracts of all relevant publications retrieved by electronic search were independently reviewed by the researchers. As a result of the search, 19165 studies (Google Scholar: 18600, Pubmed: 549, Scopus: 2, WOS: 14) were reached. The studies were firstly analyzed according to their titles, and 16625 studies that were not related to the research topic were excluded. Abstracts and full texts of the remaining 2540 studies were screened for inclusion and exclusion criteria. A total of 226 studies including reviews, letters to the editor, meta-analyses, and conference proceedings were excluded. A total of 416 duplicated articles were identified and removed using the Mendeley Reference Manager program. A total of 6 studies were found to meet the criteria for systematic review. The methodological quality of the articles included in this systematic review was evaluated by both researchers. Articles using artificial intelligence techniques in the research were accepted as diagnostic test accuracy studies. The Joanna Briggs Institute (JBI) Critical Appraisal Checklist for Diagnostic Test Accuracy Studies was used to assess the quality of the included studies. A score of 0% to 50% was considered low quality, 50% to 70% was considered medium quality, and any text article with a score of 70% and above was considered high quality. RoBvis 2 tool was used for risk of bias in the study. The decision was expressed as ‘Low’ or ‘High’ or ‘Some concerns’. The study did not require ethical approval as the research articles included in the sample were obtained from openly accessible electronic databases and search engines. All stages of the study were conducted in accordance with the principles of the Declaration of Helsinki.
Results: Of the 19165 studies initially identified, 6 were included. Six articles were critically appraised. The methodological quality of the articles was high, and all scored 70% or more. The systematic review included five studies using RF algorithm [1-5], four studies using SVM algorithm [1,4-6], DT algorithm in four studies [2,4-6], NB algorithm in four studies [3-6], LR algorithm in three studies [3-5], MLP algorithm in two studies [3,6], XGB algorithm in two studies [2,4], recursive partitioning and C-forest algorithm in one study [1], GNB, BNB and Lightgbm algorithm in one study [2], PART, J48, LogitBoost and AdaBoost algorithm in one study [3], KNN and ANN algorithm in one study [4], LASSO regression in one study [5].
Chandir et al. (2018) reported that the RF model provided 94.9% sensitivity and 54.9% specificity, while the recursive partitioning algorithm achieved the highest AUC value (0.791, 95% CI 0.784-0.798) [1]. Hasan et al. (2021) reported that the optimized LightGBM model performed best with 84.60% sensitivity and 80.0% AUC, and the performance improved with the combination of XGBoost and LightGBM [2]. Demshah et al. (2023) found that the PART algorithm gave the best results with 95.53% accuracy, followed by J48, MLP, and random forest models [3]. Tadese et al. (2024) stated that the XGBoost model stands out with 79.01% accuracy, 89.88% recall, 81.10% F1 score, 73.89% sensitivity, and 86% AUC [4]. Wang et al. (2024) emphasized that the RF model performed best on the training set, while logistic regression and Naive Bayes models stood out on the validation set [5]. Qazi et al. (2021) reported that the MLP model correctly predicted the probability of children defaulting in the immunization series with 98.5% accuracy and 0.994 AUC [6].
Discussion-Conclusion: These results show that various ML algorithms are effective in predicting childhood vaccination. However, the performance of each model varies depending on the dataset and methodology. While some algorithms, such as recursive partitioning and LightGBM, stand out, combinations and ensembles play a critical role in increasing prediction accuracy. This systematic review examines the effectiveness of ML methods in childhood vaccination prediction and provides important findings for nursing practice. The results show that algorithms such as RF, SVM, and XGBoost are particularly effective in predicting vaccination with high accuracy and sensitivity. In addition, it was determined that ensemble and combination approaches have the potential to increase vaccination prediction performance.