Reliability and validity analysis of guideline clinical applicability evaluation tools_Chinese Journal of Evidence-Based Medicine

Authors：

LI Hailong ^1,2 ^# , ZHANG Chuan ^1,2 ^# , HUANG Chao ³ , ZENG Linan ^1,2 , YANG Chunsong ^1,2 , HUANG Zongyao ^1,2,4 , YI Qiusha ^1,2,5 , XU Peipei ^1,2,5 , LIU Guanjian ⁶ , KANG Deying ⁶ , LIU Qiaolan ⁷ , ZHAO Rongsheng ^8,9 , LIU Hanmin ^2,10 , ZHANG Junhua ¹¹ , LI Youping ⁶ ,  WANG Qiang ³ ,  ZHANG Lingli ^1,2,9

1. Department of Pharmacy Evidence-Based Pharmacy Center, West China Second University Hospital, Sichuan University, Chengdu 610041, P.R.China;
2. Key Laboratory of Birth Defects and Related Diseases of Women and Children (Sichuan University), Ministry of Education, Chengdu 610041, P.R.China;
3. Medical Management Service Guidance Center, National Health Commission of the People’s Republic of China, Beijing 100044, P.R.China;
4. West China School of Pharmacy, Sichuan University, Chengdu 610041, P.R.China;
5. West China School of Medicine, Sichuan University, Chengdu 610041 P.R.China;
6. Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu 610041, P.R.China;
7. West China School of Public Health, Sichuan University, Chengdu 610041, P.R.China;
8. Department of Pharmacy, The Third Hospital of Peking University, Beijing 100191, P.R.China;
9. Evidence-Based Pharmacy Committee, Chinese Pharmaceutical Association, Beijing 100044, P.R.China;
10. Department of Pediatrics, West China Second University Hospital, Sichuan University, Chengdu 610041, P.R.China;
11. Centre for Evidence-Based Medicine, Tianjin University of TCM, Tianjin 301617, P.R.China;

Corresponding?author：

WANG Qiang, Email: jason2019@sina.cn; ZHANG Lingli, Email: zhanglingli@scu.edu.cn

Keywords：

Guideline applicability tool; Reliability; Validity; Evaluation

DOI：

10.7507/1672-2531.201909124

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objectives This study aimed to evaluate the validity and reliability of the clinical practice guidelines (CPGs) applicability evaluation tool, a preliminary revised tool, by using it to appraise specific clinical guidelines.Methods Medical staffs were sampled from relevant departments in domestic medical institutions to use tool to evaluate the two guidelines. Spearman-Brown coefficient of odd-even split-half method and Cronbach's alpha coefficient were used to evaluate the split-half reliability and internal consistency reliability. The convergent and discriminant validity were evaluated by correlation analysis and correlation coefficient comparison hypothesis test, and the structural validity was investigated by confirmatory factor analysis based on structural equation.Results The split-half reliability of the evaluation tool was 0.86, and the Cronbach's coefficient of the whole tool and each dimension were greater than 0.7 for two guidelines. The success rates of tool convergent and discriminant validity calibration were 100%. In the second-order confirmatory factor analysis model, the χ²anddf were 3.38 and 2.46, the comparative fit index (CFI) were 0.872 and 0.974, the goodness of fit index (GFI) were 0.954 and 0.983, and the adjusted goodness of fit index (AGFI) were 0.846 and 0.959 for two guidelines respectively. Both standard root mean square residual (SRMR) and root mean square error of approximation (RMSEA) were less than 0.09. Both P values of RMSEA hypothesis test were greater than 0.05.Conclusions The evaluation scale is a valid and reliable instrument for assessing the applicability of CPGs, which should be further evaluated in practical applications in the future.

Citation： LI Hailong, ZHANG Chuan, HUANG Chao, ZENG Linan, YANG Chunsong, HUANG Zongyao, YI Qiusha, XU Peipei, LIU Guanjian, KANG Deying, LIU Qiaolan, ZHAO Rongsheng, LIU Hanmin, ZHANG Junhua, LI Youping, WANG Qiang, ZHANG Lingli. Reliability and validity analysis of guideline clinical applicability evaluation tools. Chinese Journal of Evidence-Based Medicine, 2020, 20(4): 378-382. doi: 10.7507/1672-2531.201909124 Copy

1.	史靜琤, 莫顯昆, 孫振球. 量表編制中內容效度指數的應用. 中南大學學報 (醫學版), 2012, 37(2): 152-155.
2.	孫振球, 徐勇勇. 醫學統計學. 北京: 人民衛生出版社, 2014.
3.	石炳毅, 陳莉萍. 中國腎移植排斥反應臨床診療指南 (2016 版). 器官移植, 2016, 5(5): 332-338.
4.	中華醫學會糖尿病學分會. 中國 2 型糖尿病防治指南 (2017 年版). 中國實用內科雜志, 2018, 38(4): 34-86.
5.	Tabachnick BG, Fidell LS. Using multivariate statistics (5th edtion). New York: Allyn and Bacon, 2007.
6.	鄧晨卉, 劉洋, 張宇, 等. SF-12 量表在西部農村青少年中信度及效度評價. 中國公共衛生, 2016, 32(11): 1513-1516.
7.	王山, 樊文潔, 俞婉琦, 等. SF-36 量表應用于城市化居民生命質量評價的信度和效度分析. 中華流行病學雜志, 2016, 37(3): 344-347.
8.	張莎, 田晉, 劉巧蘭, 等. 流動人口 SF-12 生命質量量表信度、效度評價. 中國公共衛生, 2011, 27(2): 226-227.
9.	江梅. 總體相關系數比較的假設檢驗. 中國衛生統計, 2010, 27(1): 83-84.
10.	Li H, Xie R, Wang Y, et al. A new scale for the evaluation of clinical practice guidelines applicability: development and appraisal. Implement Sci, 2018, 13(1): 61.
11.	Collaboration A. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care, 2003, 12(1): 18-23.
12.	Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ, 2010, 182(10): 1045-1052.
13.	Brouwers MC, Kho ME, Browman GP, Burgers JS, et al. Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ, 2010, 182(10): E472-478.
14.	Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ, 2010, 182(18): E839-842.

1. 史靜琤, 莫顯昆, 孫振球. 量表編制中內容效度指數的應用. 中南大學學報 (醫學版), 2012, 37(2): 152-155.
2. 孫振球, 徐勇勇. 醫學統計學. 北京: 人民衛生出版社, 2014.
3. 石炳毅, 陳莉萍. 中國腎移植排斥反應臨床診療指南 (2016 版). 器官移植, 2016, 5(5): 332-338.
4. 中華醫學會糖尿病學分會. 中國 2 型糖尿病防治指南 (2017 年版). 中國實用內科雜志, 2018, 38(4): 34-86.
5. Tabachnick BG, Fidell LS. Using multivariate statistics (5th edtion). New York: Allyn and Bacon, 2007.
6. 鄧晨卉, 劉洋, 張宇, 等. SF-12 量表在西部農村青少年中信度及效度評價. 中國公共衛生, 2016, 32(11): 1513-1516.
7. 王山, 樊文潔, 俞婉琦, 等. SF-36 量表應用于城市化居民生命質量評價的信度和效度分析. 中華流行病學雜志, 2016, 37(3): 344-347.
8. 張莎, 田晉, 劉巧蘭, 等. 流動人口 SF-12 生命質量量表信度、效度評價. 中國公共衛生, 2011, 27(2): 226-227.
9. 江梅. 總體相關系數比較的假設檢驗. 中國衛生統計, 2010, 27(1): 83-84.
10. Li H, Xie R, Wang Y, et al. A new scale for the evaluation of clinical practice guidelines applicability: development and appraisal. Implement Sci, 2018, 13(1): 61.
11. Collaboration A. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care, 2003, 12(1): 18-23.
12. Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ, 2010, 182(10): 1045-1052.
13. Brouwers MC, Kho ME, Browman GP, Burgers JS, et al. Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ, 2010, 182(10): E472-478.
14. Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ, 2010, 182(18): E839-842.

Chinese Journal of Evidence-Based Medicine

Reliability and validity analysis of guideline clinical applicability evaluation tools

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content