The summaries are free for public
use. The Chronic Liver Disease
Foundation will continue to add and
archive summaries of articles deemed
relevant to CLDF by the Board of
Trustees and its Advisors.
Abstract Details
Clinical comparison between trial participants and potentially eligible patients using electronic health record data: A generalizability assessment method
J Biomed Inform. 2021 May 25;119:103822. doi: 10.1016/j.jbi.2021.103822. Online ahead of print.
James R Rogers1, George Hripcsak2, Ying Kuen Cheung3, Chunhua Weng4
Author information
1Department of Biomedical Informatics, Columbia University, New York, NY, United States.
2Department of Biomedical Informatics, Columbia University, New York, NY, United States; Medical Informatics Services, New York-Presbyterian Hospital, New York, NY, United States.
3Department of Biostatistics, Columbia University, New York, NY, United States.
4Department of Biomedical Informatics, Columbia University, New York, NY, United States. Electronic address: chunhua@columbia.edu.
Abstract
Objective: To present a generalizability assessment method that compares baseline clinical characteristics of trial participants (TP) to potentially eligible (PE) patients as presented in their electronic health record (EHR) data while controlling for clinical setting and recruitment period.
Methods: For each clinical trial, a clinical event was defined to identify patients of interest using available EHR data from one clinical setting during the trial's recruitment timeframe. The trial's eligibility criteria were then applied and patients were separated into two mutually exclusive groups: (1) TP, which were patients that participated in the trial per trial enrollment data; (2) PE, the remaining patients. The primary outcome was standardized differences in clinical characteristics between TP and PE per trial. A standardized difference was considered prominent if its absolute value was greater than or equal to 0.1. The secondary outcome was the difference in mean propensity scores (PS) between TP and PE per trial, in which the PS represented prediction for a patient to be in the trial. Three diverse trials were selected for illustration: one focused on hepatitis C virus (HCV) patients receiving a liver transplantation; one focused on leukemia patients and lymphoma patients; and one focused on appendicitis patients.
Results: For the HCV trial, 43 TP and 83 PE were found, with 61 characteristics evaluated. Prominent differences were found among 69% of characteristics, with a mean PS difference of 0.13. For the leukemia/lymphoma trial, 23 TP and 23 PE were found, with 39 characteristics evaluated. Prominent differences were found among 82% of characteristics, with a mean PS difference of 0.76. For the appendicitis trial, 123 TP and 242 PE were found, with 52 characteristics evaluated. Prominent differences were found among 52% of characteristics, with a mean PS difference of 0.15.
Conclusions: Differences in clinical characteristics were observed between TP and PE among all three trials. In two of the three trials, not all of the differences necessarily compromised trial generalizability and subsets of PE could be considered similar to their corresponding TP. In the remaining trial, lack of generalizability appeared present, but may be a result of other factors such as small sample size or site recruitment strategy. These inconsistent findings suggest eligibility criteria alone are sometimes insufficient in defining a target group to generalize to. With caveats in limited scalability, EHR data quality, and lack of patient perspective on trial participation, this generalizability assessment method that incorporates control for temporality and clinical setting promise to better pinpoint clinical patterns and trial considerations.