Artificial intelligence promotes the development of automated tools for systematic reviews_Chinese Journal of Evidence-Based Medicine

Authors：

ZHENG Qingyong ^1,2,3 , ZHOU Yongjia ^1,2,3,4 , ZHANG Mengjun ⁵ , CHENG Luying ⁶ , XU Jianguo ^1,2,3 , LI Tengfei ^1,2,3,4 , LIU Ming ^1,2,3 , ZHAO Chunxiang ⁷ , DI Baoshan ^8,9 , DU Li ¹⁰ ,  YANG Fengwen ^11,12 ,  TIAN Jinhui ^1,2,3

1. Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, P. R. China;
2. Key Laboratory of Evidence-based Medicine of Gansu Province, Lanzhou 730000, P. R. China;
3. Research Unit of Evidence-Based Evaluation and Guidelines, Chinese Academy of Medical Sciences, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, P. R. China;
4. School of Nursing, Gansu University of Chinese Medicine, Lanzhou 730000, P. R. China;
5. College of Nursing, Hebei University, Baoding 071000, P. R. China;
6. Zigong First People's Hospital, Zigong 643099, P. R. China;
7. The First People's Hospital of Lanzhou City, Lanzhou 730050, P. R. China;
8. The First Clinical College of Gansu University of Chinese Medicine, Lanzhou 730000, P. R. China;
9. Department of Emergency, Shenzhen Longgang Central Hospital, Shenzhen 518116, P. R. China;
10. The Third People's Hospital of Lanzhou, Lanzhou 730050, P. R. China;
11. Evidence-Based Medicine Center, College of Traditional Chinese Medicine, Tianjin University of Traditional Chinese Medicine, Tianjin 301617, P. R. China;
12. NMPA Key Laboratory for Evidence-based Evaluation of Traditional Chinese Medicine, Tianjin 301617, P. R. China;

Corresponding?author：

YANG Fengwen, Email: 13682027022@163.com; TIAN Jinhui, Email: tjh996@163.com

Keywords：

Artificial intelligence; Natural language processing; Systematic review; Meta-analysis; Automated tools

DOI：

10.7507/1672-2531.202501115

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Systematic reviews and meta-analyses are essential methods in evidence-based medicine for integrating research evidence and guiding clinical decision-making. However, with the rapid expansion of medical research data, traditional approaches face significant challenges in terms of efficiency, accuracy, and reliability. In recent years, the rapid advancement of artificial intelligence (AI) technologies, particularly in natural language processing (NLP), machine learning (ML), and large language models (LLMs), has provided robust support for automating and intelligentizing systematic reviews and meta-analyses. This paper systematically reviews the progress of AI applications in these fields, tracing the evolution from traditional tools to intelligent platforms, and analyzes the functional characteristics, application scenarios, and limitations of existing AI-driven tools. Furthermore, it explores the challenges posed by AI in terms of adaptation to the medical field, multimodal data processing, and ethical transparency, while offering potential solutions and optimization strategies. Looking ahead, with the continuous optimization of technology, enhanced data sharing, and the establishment of industry standards, AI is expected to significantly improve the efficiency and quality of systematic reviews and meta-analyses, driving the transition from "tool-driven" to "intelligent collaboration." The deep integration of AI not only injects innovative momentum into evidence-based medicine but also reshapes its methodological foundation, laying a solid basis for a more intelligent, equitable, and efficient future.

Citation： ZHENG Qingyong, ZHOU Yongjia, ZHANG Mengjun, CHENG Luying, XU Jianguo, LI Tengfei, LIU Ming, ZHAO Chunxiang, DI Baoshan, DU Li, YANG Fengwen, TIAN Jinhui. Artificial intelligence promotes the development of automated tools for systematic reviews. Chinese Journal of Evidence-Based Medicine, 2025, 25(11): 1340-1349. doi: 10.7507/1672-2531.202501115 Copy

1.	Ouzzani M, Hammady H, Fedorowicz Z, et al. Rayyan-a web and mobile app for systematic reviews. Syst Rev, 2016, 5(1): 210.
2.	Marshall IJ, Kuiper J, Wallace BC. RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. J Am Med Inform Assoc, 2016, 23(1): 193-201.
3.	Harrison H, Griffin SJ, Kuhn I, et al. Software tools to support title and abstract screening for systematic reviews in healthcare: an evaluation. BMC Med Res Methodol, 2020, 20(1): 7.
4.	Henderson LK, Craig JC, Willis NS, et al. How to write a Cochrane systematic review. Nephrology (Carlton), 2010, 15(6): 617-624.
5.	Suurmond R, van Rhee H, Hak T. Introduction, comparison, and validation of Meta-Essentials: A free and simple tool for meta-analysis. Res Synth Methods, 2017, 8(4): 537-553.
6.	Kohl C, Mcintosh E J, Unger S, et al. Online tools supporting the conduct and reporting of systematic reviews and systematic maps: a case study on CADIMA and review of existing tools. Environ Evid, 2018, 7: 1-17.
7.	EPPI-Centre. EPPI-Reviewer 4: software for research synthesis. London: Social Science Research Unit, Institute of Education, University of London. 2014.
8.	Russell-Rose T, Gooch P. 2dSearch: A visual approach to search strategy formulation. 2018.
9.	Li J, Dada A, Puladi B, et al. ChatGPT in healthcare: a taxonomy and systematic review. Comput Methods Programs Biomed, 2024, 245: 108013.
10.	Aum S, Choe S. srBERT: automatic article classification model for systematic review using BERT. Syst Rev, 2021, 10(1): 285.
11.	Lee S, Kim D, Lee K, et al. BEST: next-generation biomedical entity search tool for knowledge discovery from biomedical literature. PLoS One, 2016, 11(10): e0164680.
12.	Grames E M, Stillman A N, Tingley M W, et al. An automated approach to identifying search terms for systematic reviews using keyword co-occurrence networks. Methods Ecol Evol, 2019, 10(10): 1645-1654.
13.	Gates A, Johnson C, Hartling L. Technology-assisted title and abstract screening for systematic reviews: a retrospective evaluation of the Abstrackr machine learning tool. Syst Rev, 2018, 7(1): 45.
14.	Marshall IJ, Noel-Storr A, Kuiper J, et al. Machine learning for identifying randomized controlled trials: an evaluation and practitioner's guide. Res Synth Methods, 2018, 9(4): 602-614.
15.	Noel-Storr A, Dooley G, Elliott J, et al. An evaluation of Cochrane crowd found that crowdsourcing produced accurate results in identifying randomized trials. J Clin Epidemiol, 2021, 133: 130-139.
16.	Howard BE, Phillips J, Tandon A, et al. SWIFT-active screener: accelerated document screening through active learning and integrated recall estimation. Environ Int, 2020, 138: 105623.
17.	Przyby?a P, Brockmeier AJ, Kontonatsios G, et al. Prioritising references for systematic reviews with RobotAnalyst: a user study. Res Synth Methods, 2018, 9(3): 470-488.
18.	Cheng SH, Augustin C, Bethel A, et al. Using machine learning to advance synthesis and use of conservation and environmental evidence. Conserv Biol, 2018, 32(4): 762-764.
19.	Yu Z, Menzies T. FAST2: an intelligent assistant for finding relevant papers. Expert Syst Appl, 2019, 120: 57-71.
20.	van der Pol JA, Huizinga TW, Bergstra SA. Is AI-assisted active learning software able to reliably speed-up systematic literature reviews in rheumatology. A real-time comparison of AI-assisted and manual abstract selection. RMD Open, 2024, 10(4): e005024.
21.	Romanov S, Siqueira AS, De Bruin J, et al. Optimizing ASReview simulations: a generic multiprocessing solution for ‘light-data’and ‘heavy-data’users . Data Intell, 2024, 1-19.
22.	Haddaway NR, Grainger MJ, Gray CT. Citationchaser: a tool for transparent and efficient forward and backward citation chasing in systematic searching. Res Synth Methods, 2022, 13(4): 533-545.
23.	Kiritchenko S, de Bruijn B, Carini S, et al. ExaCT: automatic extraction of clinical trial characteristics from journal publications. BMC Med Inform Decis Mak, 2010, 10: 56.
24.	Walker VR, Schmitt CP, Wolfe MS, et al. Evaluation of a semi-automated data extraction tool for public health literature-based reviews: Dextr. Environ Int, 2022, 159: 107025.
25.	Jap J, Saldanha IJ, Smith BT, et al. Features and functioning of data abstraction assistant, a software application for data abstraction during systematic reviews. Res Synth Methods, 2019, 10(1): 2-14.
26.	Main Finding Recognition. ASReview Documentation, 2024.
27.	Hamel C, Kelly SE, Thavorn K, et al. An evaluation of DistillerSR's machine learning-based prioritization tool for title/abstract screening - impact on reviewer-relevant outcomes. BMC Med Res Methodol, 2020, 20(1): 256.
28.	Tsou AY, Treadwell JR, Erinoff E, et al. Machine learning for screening prioritization in systematic reviews: comparative performance of Abstrackr and EPPI-Reviewer. Syst Rev, 2020, 9(1): 73.
29.	Adusumilli G, Pederson JM, Hardy N, et al. Mechanical thrombectomy with and without intravenous tissue plasminogen activator for acute ischemic stroke: a systematic review and meta-analysis using nested knowledge. Front Neurol, 2021, 12: 759759.
30.	Oami T, Okada Y, Nakada TA. Performance of a large language model in screening citations. JAMA Netw Open, 2024, 7(7): e2420496.
31.	Ronquillo JG, Ye J, Gorman D, et al. Practical aspects of using large language models to screen abstracts for cardiovascular drug development: cross-sectional study. JMIR Med Inform, 2024, 30: 12: e64143.
32.	Ghosh M, Mukherjee S, Ganguly A, et al. AlpaPICO: extraction of PICO frames from clinical trial documents using LLMs. Methods, 2024, 226: 78-88.
33.	Guo E, Gupta M, Deng J, et al. Automated paper screening for clinical reviews using large language models: data analysis study. J Med Internet Res, 2024, 26: e48996.
34.	Schopow N, Osterhoff G, Baur D. Applications of the natural language processing tool chatgpt in clinical practice: comparative study and augmented systematic review. JMIR Med Inform, 2023, 11: e48933.
35.	Lai H, Ge L, Sun M, et al. Assessing the risk of bias in randomized clinical trials with large language models. JAMA Netw Open, 2024, 7(5): e2412687.
36.	Craig DB, Dr?ghici S. LmRaC: a functionally extensible tool for LLM interrogation of user experimental results. Bioinformatics, 2024, 40(12): btae679.
37.	Ivanisenko TV, Demenkov PS, Ivanisenko VA. An accurate and efficient approach to knowledge extraction from scientific publications using structured ontology models, graph neural networks, and large language models. Int J Mol Sci, 2024, 25(21): 11811.
38.	Marshall IJ, Wallace BC. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev, 2019, 8(1): 163.
39.	郭玉杰, 張雪芹, 孫文宇, 等. 自動化文獻篩選工具在系統評價中的應用. 協和醫學雜志, 2024, 15(4): 921-926.
40.	毛渤淳, 陳圣愷, 謝雨, 等. 經典深度學習算法對中文隨機對照試驗智能判別應用. 中國循證醫學雜志, 2019, 19(11): 1262-1267.
41.	Yogarajan V, Dobbie G, Keegan TT. Debiasing large language models: research opportunities. J R Soc N Z, 2024, 55(2): 372-395.
42.	Luo X, Chen F, Zhu D, et al. Potential roles of large language models in the production of systematic reviews and meta-analyses. J Med Internet Res, 2024, 26: e56780.
43.	Yang Y, Shen H, Chen K, et al. From pixels to patients: the evolution and future of deep learning in cancer diagnostics: (trends in molecular medicine, published online December 11, 2024). Trends Mol Med, 2025, 21: S1471-S4914.
44.	Haltaufderheide J, Ranisch R. The ethics of ChatGPT in medicine and healthcare: a systematic review on Large Language Models (LLMs). NPJ Digit Med, 2024, 7(1): 183.
45.	Kalaw FGP, Baxter SL. Ethical considerations for large language models in ophthalmology. Curr Opin Ophthalmol, 2024, 35(6): 438-446.
46.	Mirzaei T, Amini L, Esmaeilzadeh P. Clinician voices on ethics of LLM integration in healthcare: a thematic analysis of ethical concerns and implications. BMC Med Inform Decis Mak, 2024, 24(1): 250.
47.	Ahn S. The transformative impact of large language models on medical writing and publishing: current applications, challenges and future directions. Korean J Physiol Pharmacol, 2024, 28(5): 393-401.
48.	Zhou Y, Li SJ, Tang XY, et al. Using ChatGPT in nursing: scoping review of current opinions. JMIR Med Educ, 2024, 10: e54297.

1. Ouzzani M, Hammady H, Fedorowicz Z, et al. Rayyan-a web and mobile app for systematic reviews. Syst Rev, 2016, 5(1): 210.
2. Marshall IJ, Kuiper J, Wallace BC. RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. J Am Med Inform Assoc, 2016, 23(1): 193-201.
3. Harrison H, Griffin SJ, Kuhn I, et al. Software tools to support title and abstract screening for systematic reviews in healthcare: an evaluation. BMC Med Res Methodol, 2020, 20(1): 7.
4. Henderson LK, Craig JC, Willis NS, et al. How to write a Cochrane systematic review. Nephrology (Carlton), 2010, 15(6): 617-624.
5. Suurmond R, van Rhee H, Hak T. Introduction, comparison, and validation of Meta-Essentials: A free and simple tool for meta-analysis. Res Synth Methods, 2017, 8(4): 537-553.
6. Kohl C, Mcintosh E J, Unger S, et al. Online tools supporting the conduct and reporting of systematic reviews and systematic maps: a case study on CADIMA and review of existing tools. Environ Evid, 2018, 7: 1-17.
7. EPPI-Centre. EPPI-Reviewer 4: software for research synthesis. London: Social Science Research Unit, Institute of Education, University of London. 2014.
8. Russell-Rose T, Gooch P. 2dSearch: A visual approach to search strategy formulation. 2018.
9. Li J, Dada A, Puladi B, et al. ChatGPT in healthcare: a taxonomy and systematic review. Comput Methods Programs Biomed, 2024, 245: 108013.
10. Aum S, Choe S. srBERT: automatic article classification model for systematic review using BERT. Syst Rev, 2021, 10(1): 285.
11. Lee S, Kim D, Lee K, et al. BEST: next-generation biomedical entity search tool for knowledge discovery from biomedical literature. PLoS One, 2016, 11(10): e0164680.
12. Grames E M, Stillman A N, Tingley M W, et al. An automated approach to identifying search terms for systematic reviews using keyword co-occurrence networks. Methods Ecol Evol, 2019, 10(10): 1645-1654.
13. Gates A, Johnson C, Hartling L. Technology-assisted title and abstract screening for systematic reviews: a retrospective evaluation of the Abstrackr machine learning tool. Syst Rev, 2018, 7(1): 45.
14. Marshall IJ, Noel-Storr A, Kuiper J, et al. Machine learning for identifying randomized controlled trials: an evaluation and practitioner's guide. Res Synth Methods, 2018, 9(4): 602-614.
15. Noel-Storr A, Dooley G, Elliott J, et al. An evaluation of Cochrane crowd found that crowdsourcing produced accurate results in identifying randomized trials. J Clin Epidemiol, 2021, 133: 130-139.
16. Howard BE, Phillips J, Tandon A, et al. SWIFT-active screener: accelerated document screening through active learning and integrated recall estimation. Environ Int, 2020, 138: 105623.
17. Przyby?a P, Brockmeier AJ, Kontonatsios G, et al. Prioritising references for systematic reviews with RobotAnalyst: a user study. Res Synth Methods, 2018, 9(3): 470-488.
18. Cheng SH, Augustin C, Bethel A, et al. Using machine learning to advance synthesis and use of conservation and environmental evidence. Conserv Biol, 2018, 32(4): 762-764.
19. Yu Z, Menzies T. FAST2: an intelligent assistant for finding relevant papers. Expert Syst Appl, 2019, 120: 57-71.
20. van der Pol JA, Huizinga TW, Bergstra SA. Is AI-assisted active learning software able to reliably speed-up systematic literature reviews in rheumatology. A real-time comparison of AI-assisted and manual abstract selection. RMD Open, 2024, 10(4): e005024.
21. Romanov S, Siqueira AS, De Bruin J, et al. Optimizing ASReview simulations: a generic multiprocessing solution for ‘light-data’and ‘heavy-data’users . Data Intell, 2024, 1-19.
22. Haddaway NR, Grainger MJ, Gray CT. Citationchaser: a tool for transparent and efficient forward and backward citation chasing in systematic searching. Res Synth Methods, 2022, 13(4): 533-545.
23. Kiritchenko S, de Bruijn B, Carini S, et al. ExaCT: automatic extraction of clinical trial characteristics from journal publications. BMC Med Inform Decis Mak, 2010, 10: 56.
24. Walker VR, Schmitt CP, Wolfe MS, et al. Evaluation of a semi-automated data extraction tool for public health literature-based reviews: Dextr. Environ Int, 2022, 159: 107025.
25. Jap J, Saldanha IJ, Smith BT, et al. Features and functioning of data abstraction assistant, a software application for data abstraction during systematic reviews. Res Synth Methods, 2019, 10(1): 2-14.
26. Main Finding Recognition. ASReview Documentation, 2024.
27. Hamel C, Kelly SE, Thavorn K, et al. An evaluation of DistillerSR's machine learning-based prioritization tool for title/abstract screening - impact on reviewer-relevant outcomes. BMC Med Res Methodol, 2020, 20(1): 256.
28. Tsou AY, Treadwell JR, Erinoff E, et al. Machine learning for screening prioritization in systematic reviews: comparative performance of Abstrackr and EPPI-Reviewer. Syst Rev, 2020, 9(1): 73.
29. Adusumilli G, Pederson JM, Hardy N, et al. Mechanical thrombectomy with and without intravenous tissue plasminogen activator for acute ischemic stroke: a systematic review and meta-analysis using nested knowledge. Front Neurol, 2021, 12: 759759.
30. Oami T, Okada Y, Nakada TA. Performance of a large language model in screening citations. JAMA Netw Open, 2024, 7(7): e2420496.
31. Ronquillo JG, Ye J, Gorman D, et al. Practical aspects of using large language models to screen abstracts for cardiovascular drug development: cross-sectional study. JMIR Med Inform, 2024, 30: 12: e64143.
32. Ghosh M, Mukherjee S, Ganguly A, et al. AlpaPICO: extraction of PICO frames from clinical trial documents using LLMs. Methods, 2024, 226: 78-88.
33. Guo E, Gupta M, Deng J, et al. Automated paper screening for clinical reviews using large language models: data analysis study. J Med Internet Res, 2024, 26: e48996.
34. Schopow N, Osterhoff G, Baur D. Applications of the natural language processing tool chatgpt in clinical practice: comparative study and augmented systematic review. JMIR Med Inform, 2023, 11: e48933.
35. Lai H, Ge L, Sun M, et al. Assessing the risk of bias in randomized clinical trials with large language models. JAMA Netw Open, 2024, 7(5): e2412687.
36. Craig DB, Dr?ghici S. LmRaC: a functionally extensible tool for LLM interrogation of user experimental results. Bioinformatics, 2024, 40(12): btae679.
37. Ivanisenko TV, Demenkov PS, Ivanisenko VA. An accurate and efficient approach to knowledge extraction from scientific publications using structured ontology models, graph neural networks, and large language models. Int J Mol Sci, 2024, 25(21): 11811.
38. Marshall IJ, Wallace BC. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev, 2019, 8(1): 163.
39. 郭玉杰, 張雪芹, 孫文宇, 等. 自動化文獻篩選工具在系統評價中的應用. 協和醫學雜志, 2024, 15(4): 921-926.
40. 毛渤淳, 陳圣愷, 謝雨, 等. 經典深度學習算法對中文隨機對照試驗智能判別應用. 中國循證醫學雜志, 2019, 19(11): 1262-1267.
41. Yogarajan V, Dobbie G, Keegan TT. Debiasing large language models: research opportunities. J R Soc N Z, 2024, 55(2): 372-395.
42. Luo X, Chen F, Zhu D, et al. Potential roles of large language models in the production of systematic reviews and meta-analyses. J Med Internet Res, 2024, 26: e56780.
43. Yang Y, Shen H, Chen K, et al. From pixels to patients: the evolution and future of deep learning in cancer diagnostics: (trends in molecular medicine, published online December 11, 2024). Trends Mol Med, 2025, 21: S1471-S4914.
44. Haltaufderheide J, Ranisch R. The ethics of ChatGPT in medicine and healthcare: a systematic review on Large Language Models (LLMs). NPJ Digit Med, 2024, 7(1): 183.
45. Kalaw FGP, Baxter SL. Ethical considerations for large language models in ophthalmology. Curr Opin Ophthalmol, 2024, 35(6): 438-446.
46. Mirzaei T, Amini L, Esmaeilzadeh P. Clinician voices on ethics of LLM integration in healthcare: a thematic analysis of ethical concerns and implications. BMC Med Inform Decis Mak, 2024, 24(1): 250.
47. Ahn S. The transformative impact of large language models on medical writing and publishing: current applications, challenges and future directions. Korean J Physiol Pharmacol, 2024, 28(5): 393-401.
48. Zhou Y, Li SJ, Tang XY, et al. Using ChatGPT in nursing: scoping review of current opinions. JMIR Med Educ, 2024, 10: e54297.

Chinese Journal of Evidence-Based Medicine

Artificial intelligence promotes the development of automated tools for systematic reviews

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content