基于机器学习算法构建颈动脉粥样硬化风险预测模型

Construction of a risk prediction model for carotid atherosclerosis based on machine learning algorithms

摘要:

目的]基于筛选最优机器学习算法模型,构建并部署shinyapps.io网络版预测体检人群颈动脉粥样硬化(CAS)患病风险的内部验证模型。 [方法]回顾性收集2024年1月至2024年12月于河北医科大学第一医院体检中心接受体检的4 096例受试者,采用简单随机抽样方法按3∶1比例将样本划分为训练集(3 072例)和验证集(1 024例),两组结局变量分布均衡(训练集45.02%,验证集44.73%,P＝0.561)。收集受试者的一般资料、体格检查指标、生活方式指标、血常规指标、肝肾功能指标及病史资料。采用LASSO回归分析(通过10折交叉验证筛选最优lambda值)筛选最佳预测因子,将其分别纳入多种机器学习算法(Logit、GNB、DT、RF、XGBoost、SVM、KNN、Bagging),通过网格搜索与交叉验证优化各模型超参数；绘制多模型ROC曲线筛选性能最优算法,调用SHAP包评估每个预测因子对模型预测结果的贡献程度,绘制不同机器学习算法校准曲线进行性能对比并计算各模型准确率,绘制DCA曲线进一步验证最优机器学习算法的性能,最终将筛选出的最优机器学习算法XGBoost部署至shinyapps.io。 [结果]在训练集中,LASSO回归分析结果显示,性别、年龄、糖尿病史、高血压史、冠心病史、天冬氨酸氨基转移酶、体重指数、腰围、大血小板比值、吸烟、收缩压、白细胞计数、中性粒细胞计数、动脉硬化指数、高密度脂蛋白胆固醇是CAS的影响因素,且各变量间不存在多重共线性(VIF值均＜10)。在训练集中,XGBoost性能最佳,其预测CAS患病风险的AUC(95%CI)、准确率、灵敏度、特异度、F1、NLR、PLR分别为0.870(0.857～0.883,P＜0.001)、78.3%、80.2%、76.7%、0.769、0.258、3.446。在验证集中,XGBoost同样性能最佳,其AUC(95%CI)、准确率、灵敏度、特异度、F1、NLR、PLR分别为0.909(0.890～0.928,P＜0.001)、82.5%、82.3%、82.7%、0.808、0.214、4.754,且XGBoost模型AUC的标准偏差最小；SHAP分析结果显示,年龄对模型预测CAS风险的贡献最大,其次为收缩压。校准曲线显示,XGBoost模型的Brier分数为0.1831,与校准效果最优的SVM模型(0.1822)基本持平,且其曲线在全概率范围内均贴近理想参考线。结合其最优的判别能力与临床净获益,该模型综合性能最佳。临床决策曲线显示,XGBoost模型在较宽的风险阈值范围内具有临床净获益。在训练集中,基于XGBoost模型构建预测CAS患病风险的计算器并部署到shinyapps.io网络,该模型预测概率的最佳切割率(根据约登指数最大化确定)为44.11%。从最新的体检人群中选取2例,将相关信息填入网络版计算器,结果显示,2例受试者的CAS患病风险分别为66.33%、11.41%,结果均落在风险阈值概率范围内,初步展现了该计算器的临床应用潜力。 [结论]本研究针对CAS患病风险纳入了相对全面的指标,成功构建出准确性高、性能优良的基于XGBoost模型的实用型CAS患病风险预测网络版计算器。需要指出的是,本研究为横断面设计,构建的模型仅用于评估CAS的患病风险,不具备时间维度上的未来预测能力,后续仍需开展多中心、前瞻性研究验证该模型的泛化能力与预测性能。

Abstract:

Aim To construct and deploy an internally validated model for predicting the risk of carotid atherosclerosis (CAS) in physical examination populations on the shinyapps.io web platform, based on screening for the optimal machine learning algorithm model. Methods Retrospectively collected data from 4 096 subjects who underwent physical examinations at the Physical Examination Center of the First Hospital of Hebei Medical University from January to December 2024. Using simple random sampling, the sample was divided into a training set (3 072 cases) and a validation set (1 024 cases) in a 3∶1 ratio. The distribution of the outcome variable was balanced between the two groups (45.02% in the training set, 44.73% in the validation set, P＝0.561). General information, physical examination indicators, lifestyle indicators, blood routine indicators, liver and kidney function indicators, and medical history of the subjects were collected. LASSO regression analysis (with optimal lambda value selected via 10-fold cross-validation) was used to screen for the best predictive factors, which were then incorporated into various machine learning algorithms (Logit, GNB, DT, RF, XGBoost, SVM, KNN, Bagging). Grid search and cross-validation were employed to optimize the hyperparameters of each model. Multiple model ROC curves were plotted to select the algorithm with the best performance.The SHAP package was used to evaluate the contribution of each predictive factor to the model's prediction results. Calibration curves of different machine learning algorithms were plotted for performance comparison, and the accuracy of each model was calculated. DCA curves were plotted to further validate the performance of the optimal machine learning algorithm. Finally, the selected optimal machine learning algorithm, XGBoost, was deployed on shinyapps.io.ResultsIn the training set, LASSO regression analysis identified gender, age, history of diabetes, history of hypertension, history of coronary heart disease, aspartate aminotransferase, body mass index, waist circumference, large platelet ratio, smoking, systolic blood pressure, white blood cell count, neutrophil count, atherosclerosis index, and high density lipoprotein cholesterol as factors influencing carotid artery stenosis (CAS). No multicollinearity was observed among variables (all VIF values＜10). In the training set, XGBoost demonstrated the best performance, with AUC (95%CI), accuracy, sensitivity, specificity, F1-score, negative likelihood ratio (NLR), and positive likelihood ratio (PLR) for predicting CAS risk being 0.870 (0.857～0.883, P＜0.001), 78.3%, 80.2%, 76.7%, 0.9,0.258, and 3.446, respectively. In the validation set, XGBoost also performed best, with AUC (95%CI), accuracy, sensitivity, specificity, F1-score, NLR, and PLR of 0.909 (0.890～0.928, P＜0.001), 82.5%, 82.3%, 82.7%, 0.8,0.214, and 4.754, respectively. Additionally, XGBoost had the smallest standard deviation in AUC. SHAP analysis revealed that age contributed the most to the model's prediction of CAS risk, followed by systolic blood pressure. Calibration curves showed that the XGBoost model had a Brier score of 0.1831, which was comparable to the SVM model with the best calibration performance (0.1822), and its curve closely aligned with the ideal reference line across the entire probability range. Considering its optimal discriminative ability and clinical net benefit, this model exhibited the best overall performance. Clinical decision curves indicated that the XGBoost model provided clinical net benefit across a wide range of risk thresholds. In the training set, a web-based calculator for predicting CAS risk was developed based on the XGBoost model and deployed on shinyapps.io. The optimal cutoff probability for the model's predicted risk (determined by maximizing Youden's index) was 44.11%. Two subjects from the latest physical examination population were selected, and their relevant information was input into the web-based calculator. The results showed CAS risk probabilities of 66.33% and 11.41% for the two subjects, both falling within the risk threshold probability range, preliminarily demonstrating the clinical application potential of this calculator. Conclusions This study incorporated relatively comprehensive indicators for CAS risk and successfully developed an accurate, high-performance, practical web-based CAS risk prediction calculator based on the XGBoost model. It should be noted that this study was a cross-sectional design; the constructed model was only for assessing CAS prevalence risk and lacks the ability for future prediction over time. Future multi-center, prospective studies are needed to validate the model's generalizability and predictive performance.