The current issue of air pollution has pushed the development of the corresponding observational air pollution studies. The World Health Organization has developed a new risk of bias (RoB) assessment instrument and a related guideline for assessing the risk of potential bias in observational air pollution studies. This study introduced the background, methods, uses, advantages and disadvantages, precautions, and usage scenarios of the RoB instrument. It is expected to provide researchers with corresponding quality evaluation tools when writing related systematic review and meta-analysis, which will also help provide reporting standards for observational air pollution studies, thereby improving the quality of studies.
The COSMIN community updated the COSMIN-RoB checklist on reliability and measurement error in 2021. The updated checklist can be applied to the assessment of all types of outcome measurement studies, including clinician-reported outcome measures (ClinPOMs), performance-basd outcome measurement instruments (PerFOMs), and laboratory values. In order to help readers better understand and apply the updated COSMIN-RoB checklist and provide methodological references for conducting systematic reviews of ClinPOMs, PerFOMs and laboratory values, this paper aimed to interpret the updated COSMIN-RoB checklist on reliability and measurement error studies.
This study aims to introduce how to use the PROBAST (prediction model risk of bias assessment tool) to evaluate risk of bias and applicability of the study of diagnostic or prognostic predictive models, including the introduction of the background, the scope of application and use of the tool. This tool mainly involves the four areas of participants, predictors, outcomes and analyses. The risk of bias in the research is evaluated through the four areas, while the applicability is evaluated in the first three. PROBAST provides a standardized approach to evaluate the critical appraisal of the study of diagnostic or prognostic predictive models, which screens qualified literature for data analysis and helps to establish a scientific basis for clinical decision-making.
Selective non-reporting and publication bias of study results threaten the validity of systematic reviews and meta-analyses, thus affect clinical decision making. There are no rigorous methods to evaluate the risk of bias in network meta-analyses currently. This paper introduces the main contents of ROB-MEN (risk of bias due to missing evidence in network meta-analysis), including tables of the tool, operation process and signal questions. The pairwise comparisons table and the ROB-MEN table are the tool’s core. The ROB-MEN tool can be applied to very large and complex networks including lots of interventions to avoid time-consuming and labor-intensive process, and it has the advantages of clear logic, complete details and good applicability. It is the first tool used to evaluate the risk of bias due to missing evidence in network meta-analysis and is useful to researchers, thus being worth popularizing and applying.
ObjectiveTo interpret ROBIS, a new tool to evaluate the risk of bias in systematic reviews, to promote the comprehension of it and its proper application. MethodsWe explained each item of ROBIS tool, used it to evaluate the risk of bias of a selected intervention review whose title was Cyclophosphamide for Primary Nephrotic Syndrome of Children: A Systematic Review, and judged the risk of bias in the review. ResultsThe selected systematic review as a whole was rated as “high risk of bias”, because there existed high risk of bias in domain 2 to 4, namely identification and selection of studies, data collection and study appraisal, synthesis and findings. The risk of bias in domain 1 (study eligibility criteria) was low. The relevance of identified studies and the review’s research question was appropriately considered and the reviewers avoided emphasizing results on the basis of their statistical significance. ConclusionROBIS is a new tool worthy of being recommended to evaluate risk of bias in systematic reviews. Reviewers should use ROBIS items as standards to conduct and produce high quality systematic reviews.
ObjectiveTo evaluate the risk of bias and reliability of conclusions of systematic reviews (SRs) of lung cancer screening. MethodsWe searched PubMed, EMbase, The Cochrane Library (Issue 2, 2016), Web of Knowledge, CBM, WanFang Data and CNKI to collect SRs of lung cancer screening from inception to February 29th, 2016. The ROBIS tool was applied to assess the risk of bias of included SRs, and then GRADE system was used for evidence quality assessment of outcomes of SRs. ResultsA total of 11 SRs involving 5 outcomes (mortality, detection rate, survival rate, over-diagnosis and potential benefits and harms) were included. The results of risk of bias assessment by ROBIS tool showed:Two studies completely matched the 4 questions of phase 1. In the phase 2, 6 studies were low risk of bias in the including criteria field; 8 studies were low risk of bias in the literature search and screening field; 3 studies were low risk of bias in the data abstraction and quality assessment field; and 5 studies were low risk of bias in the data synthesis field. In the phase 3 of comprehensive risk of bias results, 5 studies were low risk. The results of evidence quality assessment by GRADE system showed:three studies had A level evidence on the outcome of mortality; 1 study had A level evidence on detection; 1 study had A level evidence on survival rate; 3 studies on over-diagnosis had C level evidence; and 2 studies on potential benefits and harms had B level evidence. ConclusionThe risk of bias of SRs of lung cancer screening is totally modest; however, the evidence quality of outcomes of these SRs is totally low. Clinicians should cautiously use these evidence to make decision based on local situation.
Nonrandomized studies are an important method for evaluating the effects of exposures (including environmental, occupational, and behavioral exposures) on human health. Risk of bias in nonrandomized studies of exposures (ROBINS-E) is used to evaluate the risk of bias in natural or occupational exposure observational studies. This paper introduces the main contents of ROBINS-E 2022, including backgrounds, seven domains, signal questions and the operation process.
Evidence synthesis is the process of systematically gathering, analyzing, and integrating available research evidence. The quality of evidence synthesis depends on the quality of the original studies included. Validity assessment, also known as risk of bias assessment, is an essential method for assessing the quality of these original studies. Currently, there are numerous validity assessment tools available, but some of them lack a rigorous development process and evaluation. The application of inappropriate validity assessment tools to assessing the quality of the original studies during the evidence synthesis process may compromise the accuracy of study conclusions and mislead the clinical practice. To address this dilemma, the LATITUDES Network, a one-stop resource website for validity assessment tools, was established in September 2023, led by academics at the University of Bristol, U.K. This Network is dedicated to collecting, sorting and promoting validity assessment tools to improve the accuracy of original study validity assessments and increase the robustness and reliability of the results of evidence synthesis. This study introduces the background of the establishment of the LATITUDES Network, the included validity assessment tools, and the training resources for the use of validity assessment tools, in order to provide a reference for domestic scholars to learn more about the LATITUDES Network, to better use the appropriate validity assessment tools to conduct study quality assessments, and to provide references for the development of validity assessment tools.
The risk of bias assessment tool 2.0 (RoB 2.0) for analyzing cluster randomized trials and crossover trials (revised version 2021) has been updated. The current paper briefly delineates the history of the RoB 2.0 tool and includes an explanation and interpretation of the updated contents and software operation process for use with cluster randomized trials and crossover trials. Compared with the previous versions, the updated RoB 2.0 tool (revised version 2021) has the advantage of precise language and is easily understood. Thus, the updated RoB 2.0 tool merits popularization and further general application.
Measurement properties studies of patient-reported outcome measures (PROMs) aims to validate the measurement properties of PROMs. In the process of designing and statistical analysis of these measurement properties studies, bias will occur if there are any defects, which will affect the quality of PROMs. Therefore, the COSMIN (consensus-based standards for the selection of health measurement instruments) team has developed the COSMIN risk of bias (COSMIN-RoB) checklist to evaluate risk of bias of studies on measurement properties of PROMs. The checklist can be used to develop systematic reviews of PROMs measurement properties, and for PROMs developers, it can also be used to guide the research design in the measurement tool development process for reducing bias. At present, similar assessment tools are lacking in China. Therefore, this article aims to introduce the primary contents of COSMIN-RoB checklist and to interpret how to evaluate risk of bias of the internal structure studies of PROMs with examples.