Assessment of Real World Observational Studies (ArRoWS) is a tool developed by the Leicester Real World Evidence (LRWE) Unit of the Diabetes Research Centre of the University of Leicester in the United Kingdom to assess the quality of real world evidence research, and has been reported to have good practicability. ArRoWS can be used to quickly and specifically assess the quality of real world evidence research that uses electronic health record information. The tool contains 16 items, nine of which are common items, and seven of which are related to specific research designs. The current study introduces the development background, development process, assessment items, assessment criteria, and application methods of ArRoWS and other related aspects, to provide references for real world researchers in China.
Objective To systematically investigate the implementation and reporting quality of statistical analysis methods in observational studies for the clinical evaluation of heart failure treatment and management devices, and to provide references for the standardized design and reporting of statistical analyses in future studies within this field. Methods A comprehensive search was conducted in the PubMed database for observational studies published between October 2014 and September 2024 that aimed to evaluate the effectiveness and/or safety of heart failure treatment devices with a control group. Two researchers independently screened the literature and extracted data. The basic characteristics of the included studies and the implementation and reporting features of their statistical analysis methods were analyzed. Results A total of 65 studies were included, comprising 63 (96.92%) cohort studies and 2 (3.08%) case-control studies. Among these, only 39 (60.00%) studies performed multivariable analyses. The median number of confounders included was 9 (IQR 5 to 16), and only 22 (56.41%) studies reported specific methods for identifying confounders. None of the studies considered procedure-related confounders such as operator experience or institutional procedure volume. The most frequently used multivariable method was Cox regression (20, 51.28%), followed by propensity score methods (13, 33.33%). Only 15 (23.08%) studies conducted subgroup analyses and 11 (16.92%) performed sensitivity analyses. Compared with studies published in non-Q1 journals according to the journal citation reports (JCR), studies published in Q1 journals had larger sample sizes and higher proportions of using multivariable analysis. Conclusion Observational studies on the clinical evaluation of heart failure treatment devices exhibit notable deficiencies in the implementation of statistical analysis methods, including inadequate identification and control of confounding factors and low proportions of subgroup and sensitivity analyses. Addressing these methodological limitations in future research will be essential for generating robust, high-quality evidence to inform clinical decision-making.