Introduction to quantitative bias analysis methods and their application in real-world studies_Chinese Journal of Evidence-Based Medicine

Authors：

ZUO Jiaxin ^1,2 , ZHOU Tiantian ^1,2 , HAN Fang ¹ ,  LIAO Xing ³ ,  HU Jing ^1,2

1. Beijing Hospital of TCM, Capital Medical University, Beijing 100010, P. R. China;
2. Center for Evidence-based Chinese Medicine, Beijing Institute of TCM, Beijing 100010, P. R. China;
3. Center for Evidence-based Medicine, Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing 100700, P. R. China;

Corresponding?author：

LIAO Xing, Email: okfrom2008@hotmail.com; HU Jing, Email: hujingebm@163.com

Keywords：

Quantitative bias analysis; Real-world study; Sensitivity analysis; Residual bias; Causal association

DOI：

10.7507/1672-2531.202511171

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Conducting real-world studies inevitably faces the challenge of bias. Researchers need to employ appropriate bias analysis methods to determine whether the strength of causal associations derived from the study is distorted by potential biases. Traditional qualitative assessments and statistical methods often struggle to effectively quantify the impact of unknown or unmeasured residual biases on study conclusions. Quantitative bias analysis can address this by constructing quantitative models to quantify sources of bias (such as misclassification or unmeasured confounding) and systematically evaluate the direction and magnitude of the effect of residual biases on effect estimates after conventional analysis, thereby confirming the robustness of the estimates. This method has gained widespread international recognition for addressing key questions such as, "How much uncertainty might potential biases introduce?" and "How credible are the study conclusions in the presence of bias?" However, there is still a lack of systematic introduction to this method domestically. Therefore, this article will begin with the practical challenges of bias analysis in real-world studies and sequentially elaborate on the development, theoretical framework, implementation cases, and application of quantitative bias analysis in real-world research. It aims to systematically introduce the methodological characteristics of quantitative bias analysis, providing researchers with a methodological reference for addressing bias analysis issues.

Citation： ZUO Jiaxin, ZHOU Tiantian, HAN Fang, LIAO Xing, HU Jing. Introduction to quantitative bias analysis methods and their application in real-world studies. Chinese Journal of Evidence-Based Medicine, 2026, 26(5): 589-597. doi: 10.7507/1672-2531.202511171 Copy

1.	Sharman JE, Kosmala W. High-quality medical research requires that equipment has been validated for accuracy. Adv Clin Exp Med, 2021, 30(12): 1221-1223.
2.	Ahlbom A. Modern epidemiology, 4th edition. TL Lash, TJ VanderWeele, S Haneuse, KJ Rothman. Wolters Kluwer, 2021. Eur J Epidemiol, 2021, 36(8): 767-768.
3.	Groenwold RH, Hak E, Hoes AW. Quantitative assessment of unobserved confounding is mandatory in nonrandomized intervention studies. J Clin Epidemiol, 2009, 62(1): 22-28.
4.	Xu C, Fan S, Tian Y, et al. Investigating the impact of trial retractions on the healthcare evidence ecosystem (VITALITY Study I): retrospective cohort study. BMJ, 2025, 23: 389: e082068.
5.	Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ, 2016, 355: i4919.
6.	Seo HJ, Kim SY, Lee YJ, et al. RoBANS 2: A revised risk of bias assessment tool for nonrandomized studies of interventions. Korean J Fam Med, 2023, 44(5): 249-260.
7.	Wells GA, Wells G, Shea B, et al. The newcastle-ottawa scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2014.
8.	Barker TH, Habibi N, Aromataris E, et al. The revised JBI critical appraisal tool for the assessment of risk of bias for quasi-experimental studies. JBI Evid Synth, 2024, 22(3): 378-388.
9.	Brown JP, Hunnicutt JN, Ali MS, et al. Core concepts in pharmacoepidemiology: quantitative bias analysis. Pharmacoepidemiol Drug Saf, 2024, 33(10): e70026.
10.	秦詩如, 汪春楠, 趙厚宇, 等. 設置外部對照的單臂試驗偏倚控制方法研究進展分析. 中國食品藥品監管, 2024, (5): 42-50.
11.	MacLehose RF, Ahern TP, Lash TL, et al. The importance of making assumptions in bias analysis. Epidemiology, 2021, 32(5): 617-624.
12.	Arah OA. Bias analysis for uncontrolled confounding in the health sciences. Annu Rev Public Health, 2017, 38: 23-38.
13.	Lash TL, Fox MP, MacLehose RF, et al. Good practices for quantitative bias analysis. Int J Epidemiol, 2014, 43(6): 1969-1985.
14.	Fox MP, Lash TL, Greenland S. A method to automate probabilistic sensitivity analyses of misclassified binary variables. Int J Epidemiol, 2005, 34(6): 1370-1376.
15.	Leahy TP, Durand-Zaleski I, Sampietro-Colom L, et al. The role of quantitative bias analysis for nonrandomized comparisons in health technology assessment: recommendations from an expert workshop. Int J Technol Assess Health Care, 2023, 39(1): e68.
16.	Berkson J. Smoking and lung cancer: some observations on two recent reports. J Am Stat Assoc, 1958, 53(281): 28-38.
17.	Hill AB. The environment and disease: association or causation. Proc R Soc Med, 1965, 58(5): 295-300.
18.	Grier JB. Nonparametric indexes for sensitivity and bias: computing formulas. Psychol Bull, 1971, 75(6): 424-429.
19.	Rubin D. Estimating causal effects of treatments in experimental and observational studies. ETS Res Bull Ser, 1972, (2): 1-31.
20.	Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika, 1983, 70(1): 41-55.
21.	Greenland S. Bias in methods for deriving standardized morbidity ratio and attributable fraction estimates. Stat Med, 1984, 3(2): 131-141.
22.	Streeter AJ, Lin NX, Crathorne L, et al. Adjusting for unmeasured confounding in nonrandomized longitudinal studies: a methodological review. J Clin Epidemiol, 2017, 87: 23-34.
23.	Lipsitch M, Tchetgen TE, Cohen T. Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiology, 2010, 21(3): 383-388.
24.	Shi X, Miao W, Tchetgen ET. A selective review of negative control methods in epidemiology. Curr Epidemiol Rep, 2020, 7(4): 190-202.
25.	Scosyrev E. Identification of causal effects using instrumental variables in randomized trials with stochastic compliance. Biom J, 2013, 55(1): 97-113.
26.	Lu B, Thomson S, Blommaert S, et al. Use of instrumental variable analyses for evaluating comparative effectiveness in empirical applications of oncology: a systematic review. J Clin Oncol, 2023, 41(13): 2362-2371.
27.	Greenland S. Basic methods for sensitivity analysis of biases. Int J Epidemiol, 1996, 25(6): 1107-1116.
28.	Greenland S, Pearl J, Robins JM. Causal diagrams for epidemiologic research. Epidemiology, 1999, 10(1): 37-48.
29.	Greenland S, Brumback B. An overview of relations among causal modelling methods. Int J Epidemiol, 2002, 31(5): 1030-1037.
30.	Rothman K, Greenland S, Lash Tl. Modern epidemiology, 3rd edition. Philadelphia: Lippincott Williams & Wilkins, 2008.
31.	Howe CJ, Cole SR. Applying quantitative bias analysis to epidemiologic data: by timothy L. Lash, matthew P. Fox, and aliza K. Fink. Am J Epidemiol, 2009, 170(10): 1316-1317.
32.	Lash TL, Fink AK. Semi-automated sensitivity analysis to assess systematic errors in observational data. Epidemiology, 2003, 14(4): 451-458.
33.	VanderWeele TJ, Ding P. Sensitivity analysis in observational research: introducing the E-value. Ann Intern Med, 2017, 167(4): 268-274.
34.	Mathur MB, VanderWeele TJ. How to report E-values for meta-analyses: recommended improvements and additions to the new GRADE approach. Environ Int, 2022, 160: 107032.
35.	Haneuse S, VanderWeele TJ, Arterburn D. Using the E-value to assess the potential effect of unmeasured confounding in observational studies. JAMA, 2019, 321(6): 602-603.
36.	Sj?lander A, Greenland S. Are E-values too optimistic or too pessimistic. Both and neither! Int J Epidemiol, 2022, 51(2): 355-363.
37.	Smith LH, Mathur MB, VanderWeele TJ. Multiple-bias sensitivity analysis using bounds. Epidemiology, 2021, 32(5): 625-634.
38.	Davis LE, Banack HR, Calderon-Anyosa R, et al. Probabilistic bias analysis for exposure misclassification of household income by neighbourhood in a cohort of individuals with colorectal cancer. Int J Epidemiol, 2024, 53(6): dyae135.
39.	Banack HR, Hayes-Larson E, Mayeda ER. Monte Carlo simulation approaches for quantitative bias analysis: a tutorial. Epidemiol Rev, 2022, 43(1): 106-117.
40.	Brown JP, Hunnicutt JN, Ali MS, et al. Quantifying possible bias in clinical and epidemiological studies with quantitative bias analysis: common approaches and limitations. BMJ, 2024, 385: e076365.
41.	Bond JC, Fox MP, Wise LA, et al. Quantitative assessment of systematic bias: a guide for researchers. J Dent Res, 2023, 102(12): 1288-1292.
42.	Lash TL, Fox MP, Cooney D, et al. Quantitative bias analysis in regulatory settings. Am J Public Health, 2016, 106(7): 1227-1230.
43.	Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet, 2002, 359(9302): 248-252.
44.	Mathur MB, VanderWeele TJ. Methods to address confounding and other biases in meta-analyses: review and recommendations. Annu Rev Public Health, 2022, 43: 19-35.
45.	Hernán M, Robins J. Causal inference: what if. Boca Raton: Chapman & Hall/CRC, 2024.
46.	Thompson CA, Arah OA. Using dags to guide the translation of priors for record-level analysis of bias due to unmeasured confounding. Am J Epidemiol, 2012, 175: S73.
47.	Fox MP, MacLehose RF, Lash TL. SAS and R code for probabilistic quantitative bias analysis for misclassified binary variables and binary unmeasured confounders. Int J Epidemiol, 2023, 52(5): 1624-1633.
48.	Nicola O, Bellocco R, Greenland S. EPISENS: Stata module for basic sensitivity analysis of epidemiological results.
49.	Mathur MB, Ding P, Riddell CA, et al. Web site and R package for computing E-values. Epidemiology, 2018, 29(5): e45-e47.
50.	Lash TL, Ahern TP, Collin LJ, et al. Bias analysis gone bad. Am J Epidemiol, 2021, 190(8): 1604-1612.
51.	Fisher DP, Johnson E, Haneuse S, et al. Association between bariatric surgery and macrovascular disease outcomes in patients with type 2 diabetes and severe obesity. JAMA, 2018, 320(15): 1570-1582.
52.	Akba? KE, Hark BD. Evaluation of quantitative bias analysis in epidemiological research: a systematic review from 2010 to mid-2023. J Eval Clin Pract, 2024, 30(7): 1413-1421.
53.	Hunnicutt JN, Ulbricht CM, Chrysanthopoulou SA, et al. Probabilistic bias analysis in pharmacoepidemiology and comparative effectiveness research: a systematic review. Pharmacoepidemiol Drug Saf, 2016, 25(12): 1343-1353.
54.	Petersen JM, Ranker LR, Barnard-Mayers R, et al. A systematic review of quantitative bias analysis applied to epidemiological research. Int J Epidemiol, 2021, 50(5): 1708-1730.

1. Sharman JE, Kosmala W. High-quality medical research requires that equipment has been validated for accuracy. Adv Clin Exp Med, 2021, 30(12): 1221-1223.
2. Ahlbom A. Modern epidemiology, 4th edition. TL Lash, TJ VanderWeele, S Haneuse, KJ Rothman. Wolters Kluwer, 2021. Eur J Epidemiol, 2021, 36(8): 767-768.
3. Groenwold RH, Hak E, Hoes AW. Quantitative assessment of unobserved confounding is mandatory in nonrandomized intervention studies. J Clin Epidemiol, 2009, 62(1): 22-28.
4. Xu C, Fan S, Tian Y, et al. Investigating the impact of trial retractions on the healthcare evidence ecosystem (VITALITY Study I): retrospective cohort study. BMJ, 2025, 23: 389: e082068.
5. Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ, 2016, 355: i4919.
6. Seo HJ, Kim SY, Lee YJ, et al. RoBANS 2: A revised risk of bias assessment tool for nonrandomized studies of interventions. Korean J Fam Med, 2023, 44(5): 249-260.
7. Wells GA, Wells G, Shea B, et al. The newcastle-ottawa scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2014.
8. Barker TH, Habibi N, Aromataris E, et al. The revised JBI critical appraisal tool for the assessment of risk of bias for quasi-experimental studies. JBI Evid Synth, 2024, 22(3): 378-388.
9. Brown JP, Hunnicutt JN, Ali MS, et al. Core concepts in pharmacoepidemiology: quantitative bias analysis. Pharmacoepidemiol Drug Saf, 2024, 33(10): e70026.
10. 秦詩如, 汪春楠, 趙厚宇, 等. 設置外部對照的單臂試驗偏倚控制方法研究進展分析. 中國食品藥品監管, 2024, (5): 42-50.
11. MacLehose RF, Ahern TP, Lash TL, et al. The importance of making assumptions in bias analysis. Epidemiology, 2021, 32(5): 617-624.
12. Arah OA. Bias analysis for uncontrolled confounding in the health sciences. Annu Rev Public Health, 2017, 38: 23-38.
13. Lash TL, Fox MP, MacLehose RF, et al. Good practices for quantitative bias analysis. Int J Epidemiol, 2014, 43(6): 1969-1985.
14. Fox MP, Lash TL, Greenland S. A method to automate probabilistic sensitivity analyses of misclassified binary variables. Int J Epidemiol, 2005, 34(6): 1370-1376.
15. Leahy TP, Durand-Zaleski I, Sampietro-Colom L, et al. The role of quantitative bias analysis for nonrandomized comparisons in health technology assessment: recommendations from an expert workshop. Int J Technol Assess Health Care, 2023, 39(1): e68.
16. Berkson J. Smoking and lung cancer: some observations on two recent reports. J Am Stat Assoc, 1958, 53(281): 28-38.
17. Hill AB. The environment and disease: association or causation. Proc R Soc Med, 1965, 58(5): 295-300.
18. Grier JB. Nonparametric indexes for sensitivity and bias: computing formulas. Psychol Bull, 1971, 75(6): 424-429.
19. Rubin D. Estimating causal effects of treatments in experimental and observational studies. ETS Res Bull Ser, 1972, (2): 1-31.
20. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika, 1983, 70(1): 41-55.
21. Greenland S. Bias in methods for deriving standardized morbidity ratio and attributable fraction estimates. Stat Med, 1984, 3(2): 131-141.
22. Streeter AJ, Lin NX, Crathorne L, et al. Adjusting for unmeasured confounding in nonrandomized longitudinal studies: a methodological review. J Clin Epidemiol, 2017, 87: 23-34.
23. Lipsitch M, Tchetgen TE, Cohen T. Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiology, 2010, 21(3): 383-388.
24. Shi X, Miao W, Tchetgen ET. A selective review of negative control methods in epidemiology. Curr Epidemiol Rep, 2020, 7(4): 190-202.
25. Scosyrev E. Identification of causal effects using instrumental variables in randomized trials with stochastic compliance. Biom J, 2013, 55(1): 97-113.
26. Lu B, Thomson S, Blommaert S, et al. Use of instrumental variable analyses for evaluating comparative effectiveness in empirical applications of oncology: a systematic review. J Clin Oncol, 2023, 41(13): 2362-2371.
27. Greenland S. Basic methods for sensitivity analysis of biases. Int J Epidemiol, 1996, 25(6): 1107-1116.
28. Greenland S, Pearl J, Robins JM. Causal diagrams for epidemiologic research. Epidemiology, 1999, 10(1): 37-48.
29. Greenland S, Brumback B. An overview of relations among causal modelling methods. Int J Epidemiol, 2002, 31(5): 1030-1037.
30. Rothman K, Greenland S, Lash Tl. Modern epidemiology, 3rd edition. Philadelphia: Lippincott Williams & Wilkins, 2008.
31. Howe CJ, Cole SR. Applying quantitative bias analysis to epidemiologic data: by timothy L. Lash, matthew P. Fox, and aliza K. Fink. Am J Epidemiol, 2009, 170(10): 1316-1317.
32. Lash TL, Fink AK. Semi-automated sensitivity analysis to assess systematic errors in observational data. Epidemiology, 2003, 14(4): 451-458.
33. VanderWeele TJ, Ding P. Sensitivity analysis in observational research: introducing the E-value. Ann Intern Med, 2017, 167(4): 268-274.
34. Mathur MB, VanderWeele TJ. How to report E-values for meta-analyses: recommended improvements and additions to the new GRADE approach. Environ Int, 2022, 160: 107032.
35. Haneuse S, VanderWeele TJ, Arterburn D. Using the E-value to assess the potential effect of unmeasured confounding in observational studies. JAMA, 2019, 321(6): 602-603.
36. Sj?lander A, Greenland S. Are E-values too optimistic or too pessimistic. Both and neither! Int J Epidemiol, 2022, 51(2): 355-363.
37. Smith LH, Mathur MB, VanderWeele TJ. Multiple-bias sensitivity analysis using bounds. Epidemiology, 2021, 32(5): 625-634.
38. Davis LE, Banack HR, Calderon-Anyosa R, et al. Probabilistic bias analysis for exposure misclassification of household income by neighbourhood in a cohort of individuals with colorectal cancer. Int J Epidemiol, 2024, 53(6): dyae135.
39. Banack HR, Hayes-Larson E, Mayeda ER. Monte Carlo simulation approaches for quantitative bias analysis: a tutorial. Epidemiol Rev, 2022, 43(1): 106-117.
40. Brown JP, Hunnicutt JN, Ali MS, et al. Quantifying possible bias in clinical and epidemiological studies with quantitative bias analysis: common approaches and limitations. BMJ, 2024, 385: e076365.
41. Bond JC, Fox MP, Wise LA, et al. Quantitative assessment of systematic bias: a guide for researchers. J Dent Res, 2023, 102(12): 1288-1292.
42. Lash TL, Fox MP, Cooney D, et al. Quantitative bias analysis in regulatory settings. Am J Public Health, 2016, 106(7): 1227-1230.
43. Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet, 2002, 359(9302): 248-252.
44. Mathur MB, VanderWeele TJ. Methods to address confounding and other biases in meta-analyses: review and recommendations. Annu Rev Public Health, 2022, 43: 19-35.
45. Hernán M, Robins J. Causal inference: what if. Boca Raton: Chapman & Hall/CRC, 2024.
46. Thompson CA, Arah OA. Using dags to guide the translation of priors for record-level analysis of bias due to unmeasured confounding. Am J Epidemiol, 2012, 175: S73.
47. Fox MP, MacLehose RF, Lash TL. SAS and R code for probabilistic quantitative bias analysis for misclassified binary variables and binary unmeasured confounders. Int J Epidemiol, 2023, 52(5): 1624-1633.
48. Nicola O, Bellocco R, Greenland S. EPISENS: Stata module for basic sensitivity analysis of epidemiological results.
49. Mathur MB, Ding P, Riddell CA, et al. Web site and R package for computing E-values. Epidemiology, 2018, 29(5): e45-e47.
50. Lash TL, Ahern TP, Collin LJ, et al. Bias analysis gone bad. Am J Epidemiol, 2021, 190(8): 1604-1612.
51. Fisher DP, Johnson E, Haneuse S, et al. Association between bariatric surgery and macrovascular disease outcomes in patients with type 2 diabetes and severe obesity. JAMA, 2018, 320(15): 1570-1582.
52. Akba? KE, Hark BD. Evaluation of quantitative bias analysis in epidemiological research: a systematic review from 2010 to mid-2023. J Eval Clin Pract, 2024, 30(7): 1413-1421.
53. Hunnicutt JN, Ulbricht CM, Chrysanthopoulou SA, et al. Probabilistic bias analysis in pharmacoepidemiology and comparative effectiveness research: a systematic review. Pharmacoepidemiol Drug Saf, 2016, 25(12): 1343-1353.
54. Petersen JM, Ranker LR, Barnard-Mayers R, et al. A systematic review of quantitative bias analysis applied to epidemiological research. Int J Epidemiol, 2021, 50(5): 1708-1730.

Chinese Journal of Evidence-Based Medicine

Introduction to quantitative bias analysis methods and their application in real-world studies

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content