Development and validation of an explainable machine learning model for predicting early mortality in patients with severe acute pancreatitis: a retrospective cohort study_CHINESE JOURNAL OF BASES AND CLINICS IN GENERAL SURGERY

Authors：

GUO Jiulin ¹ , WU Youwei ² ,  PENG Wei ³

1. Department of Information, West China Hospital, Sichuan University, Chengdu 610041, P. R. China;
2. Department of Hepatobiliary Surgery, The First Affiliated Hospital of Xi’an Jiaotong University, Xi’an 710061, Shaanxi, P. R. China;
3. Division of Liver Surgery, Department of General Surgery, West China Hospital, Sichuan University, Chengdu 610041, P. R. China;

Corresponding?author：

PENG Wei, Email: pengwei@wchscu.cn

Keywords：

severe acute pancreatitis; machine learning; in-hospital mortality; Shapley additive explanation

DOI：

10.7507/1007-9424.202507045

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective This study aimed to develop early mortality risk prediction models for patients with severe acute pancreatitis (SAP) based on eight machine learning algorithms, and to identify the major risk factors. Methods Clinical data of SAP patients diagnosed at West China Hospital of Sichuan University between January 2020 and August 2023, were retrospectively collected and randomly divided into a training set (n=878) and a validation set (n=376) in a 7∶3 ratio. Eight machine learning algorithms, including random forest, logistic regression, support vector machine, multilayer perceptron, XGBoost, Gaussian naive Bayes, CatBoost, and AdaBoost, were applied to construct early mortality prediction models for SAP. The models were evaluated using the area under curve (AUC), decision curve analysis (DCA), and Shapley additive explanations (SHAP). Results A total of 1 254 SAP patients were finally included in this study, with an early mortality rate of 15.79% (198/1 254). The random forest algorithm demonstrated the best predictive performance in both the training and validation sets, with AUCs of 0.913 and 0.844, respectively. In the DCA, random forest also yielded the greatest net benefit. SHAP analysis ranked seven key predictors of early mortality in SAP by importance: age, body mass index, heart rate, need for assisted ventilation, hemoglobin, interleukin-6, and lactate dehydrogenase, with the need for assisted ventilation being the most critical predictor. Conclusion The random forest model developed in this study can assist clinicians in more accurately identifying high-risk SAP patients at an early stage, thereby enabling timely interventions to reduce early mortality.

1.	S?reide K, Barreto SG, Pandanaboyana S. Severe acute pancreatitis. Br J Surg, 2024, 111(8): znae170. doi: 10.1093/bjs/znae170.
2.	van Dijk SM, Hallensleben NDL, van Santvoort HC, et al. Acute pancreatitis: recent advances through randomised trials. Gut, 2017, 66(11): 2024-2032.
3.	Wolbrink DRJ, van de Poll MCG, Termorshuizen F, et al. Trends in early and late mortality in patients with severe acute pancreatitis admitted to ICUs: a nationwide cohort study. Crit Care Med, 2022, 50(10): 1513-1521.
4.	Mounzer R, Langmead CJ, Wu BU, et al. Comparison of existing clinical scoring systems to predict persistent organ failure in patients with acute pancreatitis. Gastroenterology, 2012, 142(7): 1476-1482.
5.	Lepp?niemi A, Tolonen M, Tarasconi A, et al. 2019 WSES guidelines for the management of severe acute pancreatitis. World J Emerg Surg, 2019, 14: 27. doi: 10.1186/s13017-019-0247-0.
6.	Hong W, Lillemoe KD, Pan S, et al. Development and validation of a risk prediction score for severe acute pancreatitis. J Transl Med, 2019, 17(1): 146. doi: 10.1186/s12967-019-1903-6.
7.	Wu Q, Wang J, Qin M, et al. Accuracy of conventional and novel scoring systems in predicting severity and outcomes of acute pancreatitis: a retrospective study. Lipids Health Dis, 2021, 20(1): 41. doi: 10.1186/s12944-021-01470-4.
8.	Bang CS, Ahn JY, Kim JH, et al. Establishing machine learning models to predict curative resection in early gastric cancer with undifferentiated histology: development and usability study. J Med Internet Res, 2021, 23(4): e25053. doi: 10.2196/25053.
9.	Goh KH, Wang L, Yeow AYK, et al. Artificial intelligence in sepsis early prediction and diagnosis using unstructured data in healthcare. Nat Commun, 2021, 12(1): 711. doi: 10.1038/s41467-021-20910-4.
10.	Xu F, Chen X, Li C, et al. Prediction of multiple organ failure complicated by moderately severe or severe acute pancreatitis based on machine learning: a multicenter cohort study. Mediators Inflamm, 2021, 2021: 5525118. doi: 10.1155/2021/5525118.
11.	杜奕奇, 陳其奎, 李宏宇, 等. 中國急性胰腺炎診治指南 (2019年, 沈陽). 臨床肝膽病雜志, 2019, 35(12): 2706-2711.
12.	Banks PA, Bollen TL, Dervenis C, et al. Classification of acute pancreatitis-2012: revision of the Atlanta classification and definitions by international consensus. Gut, 2013, 62(1): 102-111.
13.	Xiao AY, Tan ML, Wu LM, et al. Global incidence and mortality of pancreatic diseases: a systematic review, meta-analysis, and meta-regression of population-based cohort studies. Lancet Gastroenterol Hepatol, 2016, 1(1): 45-55.
14.	夏慶, 鄧力琿, 杜丹, 等. 中西醫結合治療重癥急性胰腺炎的難點與展望. 中國普外基礎與臨床雜志, 2024, 31(2): 129-133.
15.	王東東, 魏豐賢, 張宇浩, 等. 血常規相關指標對急性胰腺炎病情嚴重程度預測價值的研究現狀. 中國普外基礎與臨床雜志, 2022, 29(2): 264-269.
16.	Matta B, Gougol A, Gao X, et al. Worldwide variations in demographics, management, and outcomes of acute pancreatitis. Clin Gastroenterol Hepatol, 2020, 18(7): 1567-1575. e2.
17.	Boxhoorn L, Voermans RP, Bouwense SA, et al. Acute pancreatitis. Lancet, 2020, 396(10252): 726-734.
18.	Li YL, Zhang DD, Xiong YY, et al. Development and external validation of models to predict acute respiratory distress syndrome related to severe acute pancreatitis. World J Gastroenterol, 2022, 28(19): 2123-2136.
19.	Basnayake C, Ratnam D. Blood tests for acute pancreatitis. Aust Prescr, 2015, 38(4): 128-130.
20.	Sternby H, Hartman H, Johansen D, et al. IL-6 and CRP are superior in early differentiation between mild and non-mild acute pancreatitis. Pancreatology, 2017, 17(4): 550-554.
21.	Tian F, Li H, Wang L, et al. The diagnostic value of serum C-reactive protein, procalcitonin, interleukin-6 and lactate dehydrogenase in patients with severe acute pancreatitis. Clin Chim Acta, 2020, 510: 665-670.
22.	Desai AD, Chinta S, Yeh C, et al. An analysis of lactate dehydrogenase (LDH) levels in advanced stage Ⅳ melanoma of the skin: prognostic capabilities and demographic variability. Arch Dermatol Res, 2023, 315(4): 799-806.
23.	Wang X, Li C, Li M, et al. Clinical significance of serum lactate and lactate dehydrogenase levels for disease severity and clinical outcomes in patients with colorectal cancer admitted to the intensive care unit. Heliyon, 2023, 10(1): e23608. doi: 10.1016/j.heliyon.2023.e23608.
24.	Comandatore A, Franczak M, Smolenski RT, et al. Lactate Dehydrogenase and its clinical significance in pancreatic and thoracic cancers. Semin Cancer Biol, 2022, 86(Pt 2): 93-100.
25.	Chen CC, Wang SS, Chao Y, et al. C-reactive protein and lactate dehydrogenase isoenzymes in the assessment of the prognosis of acute pancreatitis. J Gastroenterol Hepatol, 1992, 7(4): 363-366.
26.	Ueda T, Takeyama Y, Yasuda T, et al. Lactate dehydrogenase-to-lymphocyte ratio for the prediction of infection in acute necrotizing pancreatitis. Pancreas, 2007, 35(4): 378-380.
27.	唐豪佑, 劉勝, 胡程俊, 等. 新血液生化評分系統對急性胰腺炎嚴重程度及死亡的預測價值. 中國普外基礎與臨床雜志, 2021, 28(8): 1039-1044.
28.	Janitza S, Strobl C, Boulesteix AL. An AUC-based permutation variable importance measure for random forests. BMC Bioinformatics, 2013, 14: 119. doi: 10.1186/1471-2105-14-119.
29.	Hou F, Zhu Y, Zhao H, et al. Development and validation of an interpretable machine learning model for predicting the risk of distant metastasis in papillary thyroid cancer: a multicenter study. EClinicalMedicine, 2024, 77: 102913. doi: 10.1016/j.eclinm.2024.102913.
30.	Beigmohammadi MT, Amoozadeh L, Rezaei Motlagh F, et al. Mortality predictive value of APACHE Ⅱ and SOFA scores in COVID-19 patients in the intensive care unit. Can Respir J, 2022, 2022: 5129314. doi: 10.1155/2022/5129314.
31.	Al-Hadeedi S, Fan ST, Leaper D. APACHE-Ⅱ score for assessment and monitoring of acute pancreatitis. Lancet, 1989, 2(8665): 738. doi: 10.1016/s0140-6736(89)90795-2.
32.	Lambden S, Laterre PF, Levy MM, et al. The SOFA score-development, utility and challenges of accurate assessment in clinical trials. Crit Care, 2019, 23(1): 374. doi: 10.1186/s13054-019-2663-7.
33.	Yang CJ, Chen J, Phillips AR, et al. Predictors of severe and critical acute pancreatitis: a systematic review. Dig Liver Dis, 2014, 46(5): 446-451.
34.	Denisko D, Hoffman MM. Classification and interaction in random forests. Proc Natl Acad Sci U S A, 2018, 115(8): 1690-1692.

1. S?reide K, Barreto SG, Pandanaboyana S. Severe acute pancreatitis. Br J Surg, 2024, 111(8): znae170. doi: 10.1093/bjs/znae170.
2. van Dijk SM, Hallensleben NDL, van Santvoort HC, et al. Acute pancreatitis: recent advances through randomised trials. Gut, 2017, 66(11): 2024-2032.
3. Wolbrink DRJ, van de Poll MCG, Termorshuizen F, et al. Trends in early and late mortality in patients with severe acute pancreatitis admitted to ICUs: a nationwide cohort study. Crit Care Med, 2022, 50(10): 1513-1521.
4. Mounzer R, Langmead CJ, Wu BU, et al. Comparison of existing clinical scoring systems to predict persistent organ failure in patients with acute pancreatitis. Gastroenterology, 2012, 142(7): 1476-1482.
5. Lepp?niemi A, Tolonen M, Tarasconi A, et al. 2019 WSES guidelines for the management of severe acute pancreatitis. World J Emerg Surg, 2019, 14: 27. doi: 10.1186/s13017-019-0247-0.
6. Hong W, Lillemoe KD, Pan S, et al. Development and validation of a risk prediction score for severe acute pancreatitis. J Transl Med, 2019, 17(1): 146. doi: 10.1186/s12967-019-1903-6.
7. Wu Q, Wang J, Qin M, et al. Accuracy of conventional and novel scoring systems in predicting severity and outcomes of acute pancreatitis: a retrospective study. Lipids Health Dis, 2021, 20(1): 41. doi: 10.1186/s12944-021-01470-4.
8. Bang CS, Ahn JY, Kim JH, et al. Establishing machine learning models to predict curative resection in early gastric cancer with undifferentiated histology: development and usability study. J Med Internet Res, 2021, 23(4): e25053. doi: 10.2196/25053.
9. Goh KH, Wang L, Yeow AYK, et al. Artificial intelligence in sepsis early prediction and diagnosis using unstructured data in healthcare. Nat Commun, 2021, 12(1): 711. doi: 10.1038/s41467-021-20910-4.
10. Xu F, Chen X, Li C, et al. Prediction of multiple organ failure complicated by moderately severe or severe acute pancreatitis based on machine learning: a multicenter cohort study. Mediators Inflamm, 2021, 2021: 5525118. doi: 10.1155/2021/5525118.
11. 杜奕奇, 陳其奎, 李宏宇, 等. 中國急性胰腺炎診治指南 (2019年, 沈陽). 臨床肝膽病雜志, 2019, 35(12): 2706-2711.
12. Banks PA, Bollen TL, Dervenis C, et al. Classification of acute pancreatitis-2012: revision of the Atlanta classification and definitions by international consensus. Gut, 2013, 62(1): 102-111.
13. Xiao AY, Tan ML, Wu LM, et al. Global incidence and mortality of pancreatic diseases: a systematic review, meta-analysis, and meta-regression of population-based cohort studies. Lancet Gastroenterol Hepatol, 2016, 1(1): 45-55.
14. 夏慶, 鄧力琿, 杜丹, 等. 中西醫結合治療重癥急性胰腺炎的難點與展望. 中國普外基礎與臨床雜志, 2024, 31(2): 129-133.
15. 王東東, 魏豐賢, 張宇浩, 等. 血常規相關指標對急性胰腺炎病情嚴重程度預測價值的研究現狀. 中國普外基礎與臨床雜志, 2022, 29(2): 264-269.
16. Matta B, Gougol A, Gao X, et al. Worldwide variations in demographics, management, and outcomes of acute pancreatitis. Clin Gastroenterol Hepatol, 2020, 18(7): 1567-1575. e2.
17. Boxhoorn L, Voermans RP, Bouwense SA, et al. Acute pancreatitis. Lancet, 2020, 396(10252): 726-734.
18. Li YL, Zhang DD, Xiong YY, et al. Development and external validation of models to predict acute respiratory distress syndrome related to severe acute pancreatitis. World J Gastroenterol, 2022, 28(19): 2123-2136.
19. Basnayake C, Ratnam D. Blood tests for acute pancreatitis. Aust Prescr, 2015, 38(4): 128-130.
20. Sternby H, Hartman H, Johansen D, et al. IL-6 and CRP are superior in early differentiation between mild and non-mild acute pancreatitis. Pancreatology, 2017, 17(4): 550-554.
21. Tian F, Li H, Wang L, et al. The diagnostic value of serum C-reactive protein, procalcitonin, interleukin-6 and lactate dehydrogenase in patients with severe acute pancreatitis. Clin Chim Acta, 2020, 510: 665-670.
22. Desai AD, Chinta S, Yeh C, et al. An analysis of lactate dehydrogenase (LDH) levels in advanced stage Ⅳ melanoma of the skin: prognostic capabilities and demographic variability. Arch Dermatol Res, 2023, 315(4): 799-806.
23. Wang X, Li C, Li M, et al. Clinical significance of serum lactate and lactate dehydrogenase levels for disease severity and clinical outcomes in patients with colorectal cancer admitted to the intensive care unit. Heliyon, 2023, 10(1): e23608. doi: 10.1016/j.heliyon.2023.e23608.
24. Comandatore A, Franczak M, Smolenski RT, et al. Lactate Dehydrogenase and its clinical significance in pancreatic and thoracic cancers. Semin Cancer Biol, 2022, 86(Pt 2): 93-100.
25. Chen CC, Wang SS, Chao Y, et al. C-reactive protein and lactate dehydrogenase isoenzymes in the assessment of the prognosis of acute pancreatitis. J Gastroenterol Hepatol, 1992, 7(4): 363-366.
26. Ueda T, Takeyama Y, Yasuda T, et al. Lactate dehydrogenase-to-lymphocyte ratio for the prediction of infection in acute necrotizing pancreatitis. Pancreas, 2007, 35(4): 378-380.
27. 唐豪佑, 劉勝, 胡程俊, 等. 新血液生化評分系統對急性胰腺炎嚴重程度及死亡的預測價值. 中國普外基礎與臨床雜志, 2021, 28(8): 1039-1044.
28. Janitza S, Strobl C, Boulesteix AL. An AUC-based permutation variable importance measure for random forests. BMC Bioinformatics, 2013, 14: 119. doi: 10.1186/1471-2105-14-119.
29. Hou F, Zhu Y, Zhao H, et al. Development and validation of an interpretable machine learning model for predicting the risk of distant metastasis in papillary thyroid cancer: a multicenter study. EClinicalMedicine, 2024, 77: 102913. doi: 10.1016/j.eclinm.2024.102913.
30. Beigmohammadi MT, Amoozadeh L, Rezaei Motlagh F, et al. Mortality predictive value of APACHE Ⅱ and SOFA scores in COVID-19 patients in the intensive care unit. Can Respir J, 2022, 2022: 5129314. doi: 10.1155/2022/5129314.
31. Al-Hadeedi S, Fan ST, Leaper D. APACHE-Ⅱ score for assessment and monitoring of acute pancreatitis. Lancet, 1989, 2(8665): 738. doi: 10.1016/s0140-6736(89)90795-2.
32. Lambden S, Laterre PF, Levy MM, et al. The SOFA score-development, utility and challenges of accurate assessment in clinical trials. Crit Care, 2019, 23(1): 374. doi: 10.1186/s13054-019-2663-7.
33. Yang CJ, Chen J, Phillips AR, et al. Predictors of severe and critical acute pancreatitis: a systematic review. Dig Liver Dis, 2014, 46(5): 446-451.
34. Denisko D, Hoffman MM. Classification and interaction in random forests. Proc Natl Acad Sci U S A, 2018, 115(8): 1690-1692.

CHINESE JOURNAL OF BASES AND CLINICS IN GENERAL SURGERY

Latest ArticlesDevelopment and validation of an explainable machine learning model for predicting early mortality in patients with severe acute pancreatitis: a retrospective cohort study

Abstract Full text Figures/Tables Video References Cited by

Format

Content