Federated Electronic Health Record Database via a Public-Private Partnership to Enhance Health Care Equity by Disparity Discovery and Equitable Clinical Trial Recruitment

Abstract

Despite increasingly stringent requirements from regulatory agencies, clinical trials often fail to recruit study populations representative of real-world demographics and disease prevalence and are often skewed away from racial/ethnic minorities. Consequently, data produced by such trials can result in treatment guidelines and outcome expectations that do not apply to racial/ethnic minorities, further widening health disparities. In this study, we describe a new tool, the TriNetX Diversity Lens ("Diversity Lens"), which augments the existing electronic health record querying functionality of TriNetX and allows clinical trial sponsors to rapidly evaluate the potential impact of inclusion and exclusion criteria on the eligibility rates of different racial and ethnic groups. We describe the development of Diversity Lens in collaboration with public and private stakeholders. Additionally, we feature examples of how Diversity Lens can bring to the surface insights into existing health disparities and prospectively explore the impact of study criteria on the eligibility of racial/ethnic minorities.

Disease burden, clinical outcomes, and access to high-level care vary across racial and ethnic groups, secondary to socioeconomic or cultural oppression, intergenerational geographic construction promoting segregation and oppression, and genetic and biological factors in a limited subset of case studies and disease etiologies.¹ In the early years of clinical trial design, it was thought that narrow patient selection criteria would allow for better detection of clinical efficacy as the results would not be diluted by confounding factors.² It was quickly realized that this approach led to contradictory results between clinical trials on the same intervention and between trial performance and real-world performance,^3,4 and regulatory agencies have put increasing pressure on trial sponsors to develop and execute trials whose participants reflect the diverse real-world population.⁵ Despite these new expectations, clinical trials often fail to recruit a population representative of real-world demographics and disease prevalence which skews from racial/ethnic minority populations.^6–9 As a result, data produced by such clinical trials can result in treatment guidelines and outcome expectations that do not apply to racial/ethnic minorities. This effect exacerbates existing health disparities

Examples of the clinical disparity caused by non-representative trials include the development of dosing regimens that do not account for differences in drug absorption and metabolism, resulting in either overtreatment, which can lead to higher rates of adverse drug reactions, or undertreatment, which can lead to treatment failure.^10–13 Guidelines for treatment candidacy may not be met because the biometrics used in clinical trial design do not represent the biometrics of racial/ethnic minority populations. Furthermore, under-representation of minority populations in clinical trials further erodes racial/ethnic minority trust in the health care system. Providing equitable access to clinical trials can improve the accuracy of clinical trial results and ensure access to the potential benefits of research and cutting-edge therapies.

While there are many barriers to recruiting a diverse clinical trial cohort, including medical mistrust related to historical traumas and lack of socioeconomic resources to obtain access, narrow eligibility criteria have been identified by the FDA as a critical modifiable factor.^5,13 There are numerous tools available to assist clinical trial sponsors in defining study criteria, but it remains challenging to assess both existing health disparities and the impact of individual study criteria on the eligibility of member of racial/ethnic minority groups.

TriNetX (TriNetX, LLC, Cambridge, MA) is a private data and analytics company founded and reliant on partnerships with public and private health care organizations (HCOs). Health care organizations contribute de-identified clinical data derived from electronic health records into a federated data repository which can be accessed via a web-based analytics platform. In return, members of these organizations receive clinical trial opportunities from industry partners and free access to all research services to which their institution is contributing data.

The core functionalities of the TriNetX include a query tool for identifying patient cohorts, a site-selection tool for matching HCOs to clinical trial opportunities, and an analytics suite for conducting retrospective observational research.¹⁴ The TriNetX Diversity Lens ("Diversity Lens") is a suite of new functionalities that enables academic researchers and industry clinical trial sponsors to readily evaluate the potential impact of inclusion and exclusion criteria on the eligibility rates of different racial and ethnic groups and to identify clinical trial sites with an equitably defined trial population. In this paper, we describe the collaborative process involving both public and private stakeholders that led to the development of Diversity Lens. Additionally, we feature examples of how researchers can use TriNetX to explore and identify existing health disparities and prospectively determine eligibility criteria that decrease or increase racial/ethnic minority trial recruitment.

TriNetX features applicable to addressing health disparities

Diversity Lens can be used in two places within the platform: within the Analyze Criteria tool and within the HCOs tool. Analyze Criteria creates a funnel of the terms for a cohort from most to least impactful on the total number of eligible patients. In Analyze Criteria, users can examine how individual diagnosis, procedure, medication, and lab terms alter the eligibility of patients segmented by sex, race, or ethnicity either by absolute change, percent change, or both, and quickly see the relative impact on different demographic groups. In the HCO tool, clinical trial sponsors can use Diversity Lens to determine the rough number of eligible patients at each HCO and filter each HCO site based on sex, race, and ethnicity to determine which HCOs have patient diversity that reflects their trial's recruitment targets. It should be noted that this functionality is only available to trial sponsors and is only available on Networks that do not allow for advanced analytics or exact patient counts. A flow chart of how the tool might be integrated into the clinical trial design process is presented in Figure 1.

In addition to Diversity Lens, race and ethnicity terms can be included in a query build. This allows researchers to build multiple queries with the same clinical criteria

Figure 1.

Flow chart of incorporating Diversity Lens into the clinical trial design process.

but different race and/or ethnicity criteria. The researcher can then explore health disparities between two cohorts using TriNetX's integrated statistical analytics platform. More details about the features available on the TriNetX platform can be found on the website at https://trinetx.com/.

Methods

Data source

All data presented in this paper came from the TriNetX COVID-19 Research Network or the TriNetX US Collaborative Network queried between March and April 2022. These Networks provide access to de-identified clinical data, including diagnoses, procedures, medications, laboratory values, vitals, genomic variants, and pathology from HCOs who have agreed to participate in TriNetX. The COVID-19 Research Network includes a global mixture of HCOs, whereas the US Collaborative Network includes only HCOs based in the U.S.

TriNetX operates on a federated model where the data resides on an appliance located at the contributing HCO or in a cloud-hosted instance under the control of the contributing HCO. Users primarily interact with the data through the browser-based querying functionality. Queries use clinical facts linked by Boolean operators (AND, OR, and NOT) and time-dependent relationships to identify patients who meet a series of inclusion and exclusion criteria. The clinical and demographic characteristics of these cohorts can be examined at the aggregate level. Data on these research networks can only be accessed in an aggregate form without attribution to the contributing HCO but allow for advanced analytics and return of exact patient counts. TriNetX clinical trial networks, such as the TriNetX Global Network, have reduced analytic functionality and data precision but allow for the identification of potential clinical trial sites. Regardless of network, all counts between one and 10 are reported as 10 to ensure patient privacy.

TriNetX is compliant with the Health Insurance Portability and Accountability Act (HIPAA), the U.S. federal law that protects the privacy and security of health care data, and any additional data privacy regulations applicable to the contributing HCO. TriNetX is certified to the ISO 27001:2013 standard and maintains an information security management system to ensure the protection of the health care data it has access to and to meet the requirements of the HIPAA Security Rule. Any data displayed on the TriNetX Platform in aggregate form, or any patient-level data provided in a dataset generated by the TriNetX Platform, only contain de-identified data as per the de-identification standard defined in Section §164.514(a) of the HIPAA Privacy Rule. The process by which the data are de-identified is attested to through a formal determination by a qualified expert as defined in Section §164.514(b)(1) of the HIPAA Privacy Rule.

Development process of Diversity Lens

Diversity Lens is a set of new features within the TriNetX platform that allows users to quickly evaluate race, ethnicity, and sex data at different stages of the trial/research question design process. The inception of Diversity Lens was prompted by customer research demonstrating that clinical trial sponsors were struggling to recruit representative populations and meet regulator expectations. This was followed by a review of existing data assets to identify ways that TriNetX could address the client's need by expanding analytic capabilities. Once the concept of the features to be included in Diversity Lens was developed, they were validated through user panels of both contributing HCOs and industry clinical trial sponsors. This iterative process of development and validation was repeated through the prototype design phase and the engineering build phase, resulting in a product release in late 2021. The positive feedback following the product's release has led to additional refinement and new feature addition, which has continued into 2022.

It should be noted that the TriNetX data model currently uses U.S. Census race and ethnicity categories. This means race is limited to White, Black/African American, Asian American, American Indian/Alaska Native, and Native Hawaiian/Pacific Islander, and unknown, while ethnicity is limited to Hispanic or Latino, not Hispanic or Latino, and unknown. In addition, most HCOs outside the U.S. either do not capture race and ethnicity data or report it in categories not currently supported by the TriNetX data model.

Evaluating clinical trial functionality: A theoretical example for thyroid cancer

To demonstrate how Diversity Lens can be applied during the clinical trial design phase, we created a theoretical trial and used the tool to evaluate the impact of inclusion and exclusion criteria on patients of different races using the US Collaborative Network. For our theoretical trial, a query was designed to select for females between the age of 18 and 90 with a diagnosis of follicular or papillary thyroid cancer, no active hypertension, no recent other cancer, and no recent history of acute myocardial infarction, heart failure, or unstable angina. On TriNetX, the query design looked like this:

1. 1. Have a diagnosis of thyroid cancer with a pathology confirmation of follicular or papillary carcinoma or adenocarcinoma, and

2. 2. Be female and between the ages of 18 and 90 at the time of diagnosis, and

3. 3. At least one blood pressure reading in the year preceding the first record of thyroid cancer, and

4. 4. No record of essential hypertension in the year preceding the first record of thyroid cancer, and

5. 5. No record of other malignancy in the five years preceding the first record of thyroid cancer, and

6. 6. No record of acute myocardial infarction, heart failure, or unstable angina in the five years preceding the first record of thyroid cancer.

Next, a second query was designed where requirement four was replaced by "No record of blood pressure equal to or greater than 150/90 in the year preceding the first record of thyroid cancer." The blood pressure cutoff was derived from a review of publicly available metastatic thyroid cancer trial protocols. For this example analysis we used the cut off 150/90 specified in NCT02393690; however, the specific threshold may vary based on the excepted adverse event profile of the treatment under evaluation. Subsequently, Diversity Lens was used within the Analyze Criteria tool to examine the number and percentage of patients excluded from trial eligibility as each criterion was applied and how this differed across racial groups. We evaluated how changing requirement four from a diagnosis criterion to a laboratory criterion affected eligibility.

Evaluating clinical trial functionality: A trial design replication

As a second example of how Diversity Lens interacts with clinical trial design, a query was designed to recreate the inclusion and exclusion criteria of a randomly selected, recent, real-world thyroid cancer clinical trial from clinicaltrials.gov (NCT03246958) in the US Collaborative Network. On TriNetX the query looked like:

1. 1. Have a diagnosis of thyroid cancer with a pathology confirmation of follicular, papillary, or poorly differentiated thyroid cancer AND treated with therapeutic Iodine i-131; and

2. 2. Be over the age of 18; and

3. 3. No record of elevated creatinine (>1.5 mg/dL) following the first record of thyroid cancer; and

4. 4. No record of anemia (hemoglobin <9 g/dL) following the first record of thyroid cancer; and

5. 5. No record of elevated alanine aminotransferase (≥169 U/L) or elevated aspartate aminotransferase (≥100 U/L) following the first record of thyroid cancer; and

6. 6. No record of neutropenia (<1.5*10³ neutrophils/µL) following the first record of thyroid cancer; and

7. 7. No record of thrombocytopenia (<100*10³ platelets/µL) following the first record of thyroid cancer; and

8. 8. No record of leukopenia (>2*10³ leukocytes/µL) following the first record of thyroid cancer; and

9. 9. No record of hyperbilirubinemia (>1.8 mg/dL) following the first record of thyroid cancer; and

10. 10. No record of ECOG Performance Status Score >2 following the first record of thyroid cancer.

We then plotted the percent original population for each Diversity Lens category. Plotting the outputs illustrates which race/ethnicity/sex categories were differentially excluded from the clinical trial based on eligibility category.

Evaluation of research functionality

To evaluate the disparity in tracheostomy care, access, and incident procedures in patients who have experienced COVID-19, a simple query was designed for COVID-19 and tracheostomy on the TriNetX COVID-19 Research Network. Quickly scanning this query via the Diversity Lens revealed unequal rates across races. We built additional queries stratifying by race so that odds ratios of tracheostomy within 90 days of COVID-19 diagnosis could be calculated between races using TriNetX analytics. These odds ratios were then plotted on a forest plot to describe disparities in tracheostomy rates for patients who experienced COVID-19 stratified by race.

Results

Theoretical clinical trial design

When we ran the analysis on March 23, 2022, there were 17,112 female patients in the US Collaborative Network with a record of follicular or papillary thyroid cancer who were between the ages of 18 and 90 at the time of diagnosis. Among them, 12,144 were identified as White, 1,904 as Black/African American, 644 as Asian American, 47 as American Indian/Alaska Native, and 22 as Native Hawaiian/Pacific Islander, with the remainder categorized as other/unknown. For this demonstration, the impact of additional exclusion criteria on the White and Black/African American populations was investigated.

When looking at the original study design, 22.8% (2,767/12,144) of qualifying patients were White, and only 17.3% (330/1904) were Black/African American (Figure 2). Using the Diversity Lens, no recent diagnosis of hypertension was identified as the criterion with the largest differential in percent change between patients who are Black/African American and White.

In this hypothetical trial, one concern is adverse drug reactions in patients with pre-existing cardiac comorbidities such as uncontrolled hypertension; however, we know that a diagnosis of hypertension does not necessarily mean an individual has uncontrolled hypertension. As an alternative, we replaced the hypertension diagnosis exclusion criteria with one that excluded patients who had a systolic blood pressure of at least 150 and a diastolic blood pressure of at least 90. Under this new criteria,

Figure 2.

Patient attrition of sample protocol variations in Black/African American and White patients.

Notes

^aOccurred anytime in the year preceding the first diagnosis of Follicular or papillary thyroid cancer.

^bOccurred anytime in the 5 years preceding the first diagnosis of Follicular or papillary thyroid cancer.

ANI = acute myocardial infarction; Bp = blood pressure; dx = diagnosis; Δ% = percent change: N = number.

25.4% (3,086/12,144) of patients who are White and 23.5% (448/1904) of Black/African American qualified for trial inclusion. This small change greatly reduced the disparity in study eligibility between patient diversity categories.

Example of clinical trial functionality

After plotting the raw query data (Table 1) as percent original n as stratified by Diversity Lens categories (Figure 3), we observed that people who are Asian (68% inclusion) were most excluded by these criteria, followed by men (75% inclusion). Furthermore, we saw that the largest drop-offs occurred with elevated creatinine (18% exclusion for people who are Asian) and anemia (9% exclusion for people who are Asian) criteria. This suggests that within this clinical trial paradigm, kidney disease and anemia are the two inclusion criteria which, if modified, would have the greatest impact on generating in this case a diverse and balanced clinical trial population representative of the real-world population. Because kidney disease is etiologically linked to anemia via the erythropoietin axis, it is easy to see that these two inclusion criteria are biologically linked, and thus we can rationalize why these two criteria cross-correlate. Due to low patient counts, analysis could not be conducted on patients whose race was American Indian/Alaskan Native or Native Hawaiian/Other Pacific Islander.

Example of research functionality: COVID-19 and tracheostomy rates

For COVID-19, major interventions for acute respiratory distress syndrome include respiratory ventilation and tracheostomy for long-term ventilator management. While respirators with conventional endotracheal intubation are relatively available, access to surgical treatment options for long-term respiratory failure is limited via socioeconomic and geographic access variation. Therefore, access to tracheostomy procedures and care is a potential source of health disparity. Furthermore, increased tracheostomy rates in an urban setting where access to surgeons is good could represent increased underlying population disease burden and morbidity. To address this issue, odds ratios for receiving a tracheostomy for people who carry a COVID-19 diagnosis were calculated for each Diversity Lens category (Figure 4). Key observations from this analysis include equivocal tracheostomy odds between White and Asian populations. In contrast, patients who are Black/African American had increased odds, while people who are American Indian/Alaska Native had decreased odds of tracheostomy.

Discussion

TriNetX, in collaboration with public and private HCOs, has created a platform that not only enhances study optimization but also facilitates health disparities research and the development of clinical trials with a population that better matches the demographics of those affected by the condition of interest. In this paper, we provided two examples of how Diversity Lens can illuminate which clinical trial criteria differentially affect racial/ethnic/sex diversity in clinical trial cohorts and how changing study criteria can improve clinical trial cohort diversity and therefore (theoretically) reduce downstream health care disparities. Specifically, we found that excluding patients with a record of high blood pressure as opposed to the diagnosis code for hypertension reduced the disparity between the exclusion of patients who are White and Black/African American. In the second example, renal disease and associated anemia bore out as a key disease burden

Table 1.

RAW N FOR EXCLUSION CRITERIA FROM CLINICAL TRIAL EXAMPLE #2

Figure 3.

Percent exclusion by criteria stratified by Diversity Lens categories.

Notes

ALT = alanine aminotransferase; AST = aspartate aminotransferase; Cr = creatinine; ECOG = Eastern Cooperative Oncology Group Performance Status Scale

that unbalances thyroid cancer clinical trials. By liberalizing acceptable kidney function by modifying clinical trial dosing protocols, clinical trial results and population applicability would improve. While this paper does not provide specific recommendations, the Diversity Lens tool may be useful for the development of standardized selection criteria for the recruitment of racial/ethnic minorities.

Figure 4.

Forest plot for unadjusted odds ratio of receiving a tracheostomy within 90 days of a COVID-19 diagnosis.

The utility of the TriNetX data and platform in health disparities research has already been demonstrated in several publications by external academic researchers.^15–19 Here, we provided an example where creating a simple 2 criteria cohort (COVID-19 and tracheostomy) and then subsequently applying Diversity Lens to the cohort rapidly identified racial disparity in surgical care and disease burden/morbidity. The finding that people who are Black/African American have higher odds of receiving a tracheostomy compared with people who are White suggests that there is a higher disease burden and morbidity in people who are Black/African American. The lower odds ratio of tracheostomy among people who are American Indian/Alaskan Native despite higher incidence of COVID-19 suggests difficulty with access to care.

Thoughtful clinical trial design with diverse population inclusion is only the first step in increasing the recruitment of patients who identify as members of racial/ethnic minorities. There are many further barriers to recruitment that cannot be addressed with TriNetX alone, including distrust of medical providers due to historical wrongs, a lack of understanding or awareness of clinical trial opportunities, and insufficient resources, such as transportation, stable housing, paid time off, or childcare.²⁰ Physician/medical/researcher bias may result in patients who identify as racial/ethnic minorities not being presented with trial opportunities.²¹ A holistic approach to improving clinical trial recruitment and retention involves trial optimization, site engagement, investigator training, community engagement, and patient engagement.^22,23

The current iteration of Diversity Lens has many limitations that diminish its potential efficacy at reducing health disparity. For example, the current race and ethnicity categories are limited to the categories used in the U.S. census. Many HCOs collect more nuanced demographic data; however, as these categories vary widely between institutions, creating standard categories with greater precision has proven challenging. As new institutions join TriNetX, it will be important to refine the demographic categories to accurately reflect the diversity of these communities in a meaningful way: e.g., including identification with greater than one racial/ethnic category and including patient-reported racial identities not restricted by the limited terms used by the U.S. Census. Furthermore, while TriNetX continues to expand, at the time this study was conducted, both American Indian/Alaskan Native and Native Hawaiian/Pacific Islander races are under-reported. On this topic, TriNetX is currently taking specific actions to bolster the representation of these two racial categories. TriNetX continues to work with contributing HCOs to explore how they capture race and ethnicity data and how these data might be better represented in the TriNetX data model. Additional work is needed to apply Diversity Lens to non-U.S. data sources.

TriNetX is a private company that exists because of and is strengthened by its partnerships with public and private HCOs. It also serves as a tool for building new public/private partnerships between clinical trial sponsors and potential trial sites. TriNetX's continued existence relies on meeting the needs of all stakeholders, which includes addressing health disparities affecting marginalized racial/ethnic minority communities. Pursuit of this social justice ideal is a priority for both health care providers and clinical trial sponsors alike. Current features available in TriNetX and Diversity Lens serve as a starting place to get a handle on what issues in clinical trial diversity exist. As TriNetX data sources continue to grow and the technological capabilities expand, the power of TriNetX and Diversity Lens will continue to grow and provide insight into existing health care disparities, providing the means to strategize practical solutions that close the disparity gap one clinical trial at a time.

Authors Isatu M. Kargbo, Emily E. Noss, Harry Jin, and Jessamine P. Winer-Jones were employees of TriNetX, LLC at the time the paper was written. Zachary D. Urdang, Christopher McNair, and Elizabeth E. Cottrill have no conflicts to declare.