A narrative review of TriNetX in ophthalmology: applications, advantages, and pitfalls
Introduction
Background
TriNetX, a global federated health research network established in 2014, has become increasingly utilized in academic medicine. The database comprises a global network of healthcare organizations (HCO), pharmaceutical firms, and contract research organizations (1). Authorized users have access to de-identified electronic health record (EHR) data which includes diagnoses, procedures, medications, and laboratory values for millions of patients (2). Additionally, genomic information can be accessed for over 100,000 patients. In 2022, more than 220 HCOs across 30 countries participated in the TriNetX network (3).
TriNetX was created to enhance the efficiency of clinical research and facilitate industry-academic collaboration. HCOs provide healthcare information, while industry participants provide funding (3). For industry participants, the healthcare data enables the identification and enrollment of eligible institutions and patients in clinical research. Individual researchers can also gain access to the TriNetX database through their academic institutions. With this access, researchers can conduct various study designs and analyses, either using the platform’s integrated analytics or by manually analyzing data downloaded from TriNetX.
Within academic medicine, TriNetX has become a resource that many attending physicians, resident physicians, PhDs, and medical students use to complete research projects. Although there is no current reported data on the prevalence of TriNetX utilization in ophthalmology, there has been a recent uptick in the number of TriNetX publications in academic journals.
Rationale and knowledge gap
Major ophthalmology journals, including JAMA Ophthalmology, Ophthalmology (AAO) and associated journals, American Journal of Ophthalmology, Retina, and Journal of the American Association for Pediatric Ophthalmology and Strabismus, have recently begun publishing original articles of TriNetX analyses. However, to date, there has been no comprehensive analysis of trends in TriNetX utilization within ophthalmology research, nor has there been an evaluation of the utility of TriNetX in this field. Additionally, there has been no detailed discussion of the step-by-step process for using TriNetX or the strengths and limitations of the platform in ophthalmology research.
Objective
The objective of this review is to summarize trends in TriNetX use within ophthalmology research, describe the study designs facilitated by the platform as they pertain to ophthalmology, and explain the process of utilizing TriNetX. Furthermore, this review aims to comprehensively discuss the strengths and limitations of TriNetX in ophthalmology research. We present this article in accordance with the Narrative Review reporting checklist (available at https://aes.amegroups.com/article/view/10.21037/aes-24-27/rc).
Methods
PubMed and Scopus literature searches were conducted to identify ophthalmology articles utilizing TriNetX. The search terms: “TriNetX” and “Ophthalmology” or “Eye” were used (Table 1). Included articles were English language only from the years 1994 to June 1, 2024. Twenty-one articles published prior to June 1st, 2024, were included (Table 2).
Table 1
Items | Specification |
---|---|
Date of search | June 1, 2024 |
Databases searched | PubMed, Scopus |
Search terms used | TriNetX, Ophthalmology, Eye |
Timeframe | 1994 to June 1, 2024 |
Inclusion criteria | English publication, main investigated topic or outcome of the study related to ophthalmology, TriNetX as sole methodology |
Exclusion criteria | Non-English publication, ophthalmology as a peripheral topic in the study |
Selection criteria | The authors reviewed the included papers and verified inclusion criteria and study design |
Table 2
Journal | Date of publication | Title | Study design |
---|---|---|---|
JAMA Ophthalmol | May 2023 | Risk of New Retinal Vascular Occlusion After mRNA COVID-19 Vaccination Within Aggregated Electronic Health Record Data (4) | Retrospective cohort |
J Med Virol | October 2023 | The risk assessment of uveitis after COVID-19 diagnosis: A multicenter population-based study (5) | Retrospective cohort |
Semin Ophthalmol | November 2023 | Examining the Influence of COVID-19 Infection and Pandemic Restrictions on the Risk of Corneal Transplant Rejection or Failure: A Multicenter Study (6) | Retrospective cohort |
JAMA Ophthalmol | December 2023 | Risk of Stroke, Myocardial Infarction, and Death After Retinal Artery Occlusion (7) | Retrospective cohort |
Int Ophthalmol | December 2023 | Ophthalmology procedure trends in the United States during the COVID-19 pandemic (8) | Retrospective cohort |
Front Immunol | January 2024 | Association between immune checkpoint inhibitor medication and uveitis: a population-based cohort study utilizing TriNetX database (9) | Retrospective cohort |
Ophthalmology | January 2024 | Association of Cutaneous Keloids, Hypertrophic Scarring, and Fibrosis with Risk of Postoperative Proliferative Vitreoretinopathy (10) | Retrospective cohort |
Ophthalmology | January 2024 | Association of Primary Open-Angle Glaucoma with Diabetic Retinopathy among Patients with Type 1 and Type 2 Diabetes (11) | Retrospective cohort |
JAMA Ophthalmol | January 2024 | Epidermal Growth Factor Receptor Inhibitors for Lung Cancer and the Risk of Keratitis (12) | Retrospective cohort |
Am J Ophthalmol | January 2024 | Risk of Stroke, Myocardial Infarction, Deep Vein Thrombosis, Pulmonary Embolism, and Death After Retinal Vein Occlusion (13) | Retrospective cohort |
J Atten Disord | January 2024 | Researching Eyesight Trends IN ADHD (RETINA) (14) | |
Retina | February 2024 | RETINAL VASCULAR OCCLUSION AND COVID-19 DIAGNOSIS: A Multicenter Population-Based Study (15) | Retrospective cohort |
Diabetes Therapy | February April 2024 | Cardiovascular Outcomes with Intravitreal Anti-Vascular Endothelial Growth Factor Therapy in Patients with Diabetes: A Real-World Data Analysis (16) | Retrospective cohort |
Am J Ophthalmol | February 2024 | Association of Ocular Manifestations of Marfan Syndrome With Cardiovascular Complications (17) | |
Am J Ophthalmol | April 2024 | Impact of GLP-1 Agonists and SGLT-2 Inhibitors on Diabetic Retinopathy Progression: An Aggregated Electronic Health Record Data Study (18) | Retrospective cohort |
Diabetologia | April 2024 | Risk of diabetic retinopathy and diabetic macular oedema with sodium-glucose cotransporter 2 inhibitors and glucagon-like peptide 1 receptor agonists in type 2 diabetes: a real-world data study from a global federated database (19) | Retrospective cohort |
Eye (London) | April 2024 | Incidence of ocular pathology following bariatric surgery for with morbid obesity across a large United States National Database (20) | Retrospective cohort |
Medicina (Kaunas) | April 2024 | Assessing Uveitis Risk following Pediatric Down Syndrome Diagnosis: A TriNetX Database Study (21) | Retrospective Cohort |
Journal of the American Association for Pediatric Ophthalmology and Strabismus | April 2024 | Risk of abducens nerve palsy following COVID-19 vaccination (22) | Retrospective cohort |
In Vivo | May 2024 | Risk of Keratitis and Keratopathy in Hidradenitis Suppurativa Patients: A Global Federated Health Network Analysis (23) | Retrospective cohort |
Journal of the American Association for Pediatric Ophthalmology and Strabismus | May 2024 | Associations of strabismus surgery timing in childhood with mental health: a retrospective cohort study (24) | Retrospective cohort |
A mock study was performed to illustrate the process of conducting a TriNetX study in ophthalmology, using a methodology similar to the studies reviewed. The well-documented association between cataract extraction with intraocular lens insertion and hip fracture risk was chosen to demonstrate the use of TriNetX (25). The downloadable output provided by TriNetX was described in detail.
Discussion
Mock study
A mock investigation was conducted to illustrate the process of using TriNetX in ophthalmology research, following a design similar to the studies included in this review. The analysis involved defining the study population based on specific clinical criteria and setting up the analysis by determining the index events, outcome criteria, and time frame.
In the mock study, Cohort 1 consisted of individuals 45 years and older with age-related cataracts who underwent cataract removal within 10 years of a documented diagnosis. Cohort 2 included individuals 45 years and older with age-related cataracts who did not undergo cataract removal within 10 years of diagnosis. TriNetX’s integrated analytics balances patient characteristics within cohorts using propensity score matching based on age, race, ethnicity, and sex. Researchers can define additional variables for propensity score matching to align with the goals of the analysis. For the mock study, the cohorts were additionally matched for history of syncope and collapse, osteoporosis without current pathological fracture, type 2 diabetes mellitus, blindness and low vision, and other retinal disorders (e.g., retinopathies, dystrophy, degeneration, separation, hemorrhage). Demographic data are provided both before and after propensity score matching.
The analysis setup involves defining the index event, or the point in time when each patient in the cohort enters the analysis. TriNetX excludes index events that occurred more than 20 years ago. In this mock study, the index event was defined as the documentation of cataract diagnosis, with or without cataract removal, within 10 years of diagnosis. Patients with the target outcome before the index event were excluded to ensure proper association of outcomes with the index event in the desired temporal relationship. Although this step is not required when designing a study in TriNetX, it is applicable to retrospective cohort studies and is available to researchers if their study design requires definition of a temporal relationship. The outcome was defined as hip fracture within 10 years of the index event. TriNetX accounts for patients exiting the cohort during the analysis period by removing them from the analysis after the last recorded fact in their medical record.
In this type of study design, TriNetX runs univariate analyses which are integrated into the platform and provides a downloadable document thoroughly detailing the process and results of the study. Kaplan-Meier analysis can also be performed to estimate the probability of the outcome at specified time points, with a daily time interval used in the mock study. The output summary includes the number of patients meeting the query criteria, the number of patients with the defined outcome within the specified time window, and the proportion of patients within each cohort who experienced the outcome within the defined time frame.
Trends in utilization of TriNetX in ophthalmology research
The first article identified was published in May 2023, followed by four additional articles in the same year. From January 2024 to June 2024, 16 TriNetX articles were identified, representing a significant increase in the use of TriNetX in published ophthalmology research. By evaluating the validity and trends of TriNetX usage, researchers can make informed decisions about its suitability for their research endeavors.
All identified articles followed a retrospective cohort study design, with outcomes reported as odds ratios, risk ratios, or hazard ratios provided by TriNetX’s integrated statistical analysis software. Researchers also have the option to download data for their own analyses; however, this method is less commonly utilized. Ophthalmology studies often use outcome analysis to compare cohorts with specific exposures or procedures to matched comparators without the exposure or procedure under investigation. While many of these articles link specific exposures to outcomes, others focus on the volume of procedures within specified time frames or after certain exposures, and some investigate the timing of procedures in relation to the development of other conditions (8,24). The conclusions derived from these articles emphasize the importance of identifying and managing risks following particular exposures.
Most of the articles support previous research, with many recommending further studies to draw causal conclusions between the studied factors. Future directions often suggest incorporating multidisciplinary evaluations and long-term systemic follow-ups to strengthen conclusions.
Database utilization for retrospective cohort studies
TriNetX is predominantly used to conduct retrospective cohort studies, as demonstrated by the literature search in this review. TriNetX’s “Query Builder” allows users to search for relevant procedure and diagnosis codes to establish precise study cohorts tailored to the researcher’s needs (26). After creating cohorts, users can define exclusion criteria, target outcomes, temporal relationships between exposures and outcomes, and factors to match between cohorts.
Results are reported as risk difference, risk ratio, and odds ratio with 95% confidence intervals, z-score, and p-value. Bar charts are provided to illustrate the risk of the outcome for each cohort when Kaplan-Meier analysis is employed. For the mock study, the results showed significantly lower odds of hip fracture in those who underwent cataract removal vs. matched comparators who did not (odds ratio, 0.95; 95% CI: 0.92–0.98; P=0.0026) (Figure 1). This finding is consistent with previous research (25).
Data availability
Available data includes demographics, encounter type, diagnostic codes, procedure codes, medications/vaccinations, lab results/clinical findings/vital signs, clinical findings, genomic information, and oncologic data (Table 3). To mitigate concerns regarding data heterogeneity among participating HCOs, TriNetX maps participating HCO data to standard terminologies employed by the platform (27). Data harmonization is partially achieved by mapping demographic data to HL7 administrative standards, diagnoses to ICD-9-CM and ICD-10-CM, procedures to ICD-9-CM, ICD-10-PCS, and CPT, medications to RxNorm, labs and vital signs to LOINC, and genomics data to HGNC for gene naming and HGVS for variant descriptions. To enhance interoperability of clinical data in countries not employing the ICD-10 system and to harmonize data worldwide, TriNetX has begun to map Operationen- und Prozedurenschlüssel (OPS), used in Germany, to systematized medical nomenclature for medicine-clinical terminology (SNOMED CT) which can be used internationally to communicate EHR procedural data (28). Integrated natural language processing on the TriNetX platform extracts clinical data from medical records such as physician-written clinical documentation, granting users access to unstructured data that would otherwise not be available for use (29).
Table 3
Data type | Source |
---|---|
Demographic | EHR: Electronic health record |
ADT: Admission-Discharge-Transfer | |
Encounter type | EHR |
ADT | |
Diagnostic codes | ICD-10-CM: International Classification of Diseases, Tenth Revision, Clinical Modification |
ICD-10-GM: International Classification of Diseases, Tenth Revision, German Modification | |
SNOMED CT: Systematized Nomenclature of Medicine Clinical Terms | |
Procedure codes | CPT: Current Procedural Terminology |
ICD-10-PCS: International Classification of Diseases, Tenth Revision, Procedure Coding System | |
HCPCS: Healthcare Common Procedure Coding System | |
ICD-9: International Classification of Diseases, 9th Revision | |
OPS: Operation and Procedure Classification | |
OPCS: Office of Population Censuses and Surveys (surgical procedure codes) | |
Medications/vaccinations | RxNorm: medical prescription normalized |
NDC: National Drug Code | |
ATC: Anatomical Therapeutic Chemical (drug classification system, which divides substances into groups based on the organ or system they target (ATC 1st level), and their pharmacological or therapeutic subgroup (ATC 2nd level), chemical, pharmacological or therapeutic subgroup (ATC 3rd and 4th level), and chemical substance (ATC 5th level)) | |
AEMPS: Spanish Agency of Medicines and Medical Devices | |
DM+D: Dictionary of medicines and devices | |
CNK: National Code Number | |
Lab results and vital signs | Local lab coding |
LOINC: Logical Observation Identifiers Names and Codes | |
Clinical findings | NLP: Natural Language Processing |
Genomics information | XML: Extensible Markup Language |
JSON: JavaScript Object Notation | |
CSV: Comma-separated Values | |
VCF: Virtual Contact File | |
NAACCR: North American Association of Central Cancer Registries | |
Oncologic data | NAACCR |
ICD-O: International Classification of Diseases for Oncology | |
ICD-10-CM |
Definitions from Palchuck et al. (3).
Study designs enabled by TriNetX
Due to the vast availability of medical information and user-friendly interface, TriNetX enables various study designs. Retrospective cohort studies are the most commonly performed studies in ophthalmology research at this time. Due to the availability of data and the ability to define inputs, outcomes, and temporal relationships of inputs and outcomes, case-control studies and cross-sectional studies can also be performed. Additionally, the TriNetX database is amenable to treatment comparisons, healthcare utilization studies, drug safety/pharmacovigilance studies, risk management studies, off-label use studies, and health disparities research. In the realm of industry research and industry-academic collaboration, TriNetX allows for feasibility studies wherein industry partners can assess the availability of patients for clinical trials at desired HCOs.
Strengths and limitations of TriNetX in ophthalmology research
Study design
Retrospective cohort studies have several major limitations which become more pronounced when large databases such as TriNetX are used for research. The observational nature of ophthalmology studies using TriNetX precludes the determination of causal relationships between variables, a limitation recognized by the majority of recent TriNetX publications. Furthermore, although the platform appears straightforward and user-friendly for study design, researchers must exercise caution due to various nuances that may compromise a study’s integrity.
When designing a study, researchers must carefully consider the research network, or consortium of HCOs, they choose to use, as each network has distinct strengths and limitations. For example, the Diamond Network includes third-party claims data and demographic data for over 200 million patients but was last updated in August 2020 (30). Conversely, the Global Collaborative Network, which is continually updated, poses challenges such as the exclusion of patients from countries that do not use CPT codes when researchers choose to use only CPT codes in their study design. This nuance necessitates the incorporation of additional codes like SNOMED that can easily be overlooked by inexperienced users. For example, when working in the Global Collaborative Network, if only CPT codes are used in the inclusion criteria of the study, the researcher will not include patients who would otherwise meet the inclusion criteria if they were not coded using CPT codes but are coded using SNOMED. Conversely, if only CPT codes are used in the exclusion criteria for the control group, there may still be patients included with the characteristic that the researcher is attempting to exclude if that characteristic is coded using a SNOMED code rather than CPT codes. For this reason, researchers must be keenly aware of the function and limitations of the types of available coding systems used in TriNetX. In general, when using a Network that includes patients from countries not using CPT and ICD codes, it is best to use CPT, ICD, and SNOMED codes when defining inclusion and exclusion criteria to increase the accuracy of cohorts.
Additionally, selecting an index event is a critical step that requires careful attention. The index event defines the study’s starting point, necessitating accurate EHR code determination and definition of temporal relationships between variables and the index event to answer the research question. Errors in index event selection and relationship definition may result in invalid and erroneous conclusions.
Many of the limitations associated with study design can be overcome by using the platform’s study design assistant tool which allows researchers to ask detailed questions about their study design and receive support from TriNetX professionals. By using this tool, integral parts of studies including but not limited to design, feasibility, cohort identification, and index event selection can be streamlined and altered to improve the quality and integrity of studies on the platform. Another notable strength of the platform is that it removes patients from analysis after the last fact in their record matching the study’s inclusion criteria, accounting for patients exiting their cohorts during the analysis period which further improves study quality. The TriNetX platform also allows researchers to apply long follow-up periods, which may not be possible in other research settings. Lastly, due to the breadth of data available on the database, researchers can design studies to validate prior smaller studies, explore novel associations previously hypothesized in the literature, and analyze rare outcomes.
Coding and data availability
Data accuracy and completeness
One of the most significant limitations of TriNetX is the accuracy and completeness of its data, as not all participating organizations employ standardized methods of data collection and entry (12). This issue is particularly pertinent in ophthalmology, where many conditions are unreliably coded. Conditions that are notoriously unreliably coded include disorders of refraction and accommodation, blindness and low vision, and orbital and external diseases, all of which may be important factors to include in propensity score matching (31). Additionally, studies have demonstrated that over 50% of ophthalmic procedures may be miscoded, with an additional 10% or more being incompletely coded, likely influencing the data available on TriNetX (32). Reliance on billing codes was the most frequently recognized limitation among publications in ophthalmology (6-8,10,11,13,15,17,18-21,23,24). Due to this reliance, authors describe the inability to confirm accurate coding, accurate diagnosis, data completeness, and control for inter-organization differences in coding. Furthermore, limitations related to the ICD-10 coding system may result in misclassification bias (5,7).
Undefined pathologies
The ICD system does not comprehensively define all ophthalmologic pathology. Despite this, the platform can be used to study pathology not specifically coded by the ICD system by creatively defining cohorts based on diagnostic criteria of specific diseases where researchers can combine ICD-10 codes to create cohorts assumed to have specific target diagnoses. Many ophthalmology studies also depend on visual acuity data, which is not available on TriNetX. Consequently, studies cannot control for baseline differences in visual acuity, potentially confounding results. This limitation may affect the clinical significance of observed relationships, where statistically significant results may not reflect meaningful visual benefits (10). A similar scenario applies to studies that may depend on intraocular pressure data which, like visual acuity data, is not available. Additionally, due to the nature of the CPT coding system, researchers are unable to specify the laterality of interventions which impacts outcomes dependent on whether unilateral or bilateral interventions are performed. While ICD-10-PCS codes report laterality and are available on TriNetX, they represent inpatient populations and do not capture the majority of ophthalmology procedures.
Granularity of data
The data from TriNetX has limited granularity when not downloaded and independently analyzed, limiting the ability to measure the effect of patient-specific factors and health history on outcomes (11,13-15,19,24). Furthermore, the lack of individual-level data in this context prevents multivariate analysis that could strengthen conclusions (19). However, when users choose to download the data provided by TriNetX to perform their own statistical analyses, greater granularity is achieved, circumventing this limitation. Additionally, when researchers choose to download data for their own analyses, multivariate analysis can be performed to improve the strength of conclusions drawn from studies.
Assumption of location of ophthalmic care
HCOs participating in TriNetX cannot be assumed to be the primary source of ophthalmic care for patients being entered into the database, leading to limitations regarding the completeness and accuracy of data specific to ophthalmic care. This issue arises when analyzing the effect of ophthalmic procedures or conditions on outcomes. While outcomes may be reliably coded by organizations, ophthalmic procedures, and conditions may not be adequately or accurately accounted for. The coding accuracy of ophthalmic care may also be compromised by documentation of outside-organization ophthalmic diagnoses and procedures by non-ophthalmology providers at TriNetX-affiliated HCOs who are unfamiliar with ophthalmic conditions and coding standards (14). Further, non-ophthalmology providers may not prioritize documenting such conditions in the acute setting, resulting in incomplete, incorrect, or absent documentation of the ophthalmic topic targeted by researchers.
When designing studies dependent on the presence of ophthalmologic data in EHRs, the assumption must be made that individuals not coded as having an ophthalmic procedure or condition truly did not have the procedure. This may result in misclassification bias wherein uncaptured patients who otherwise meet selection criteria due to procedures and treatments completed by ophthalmology providers not affiliated with TriNetX are not included in the analysis. There is also a chance that patients who received ophthalmic procedures at non-participating organizations may be included in the control group, skewing results. The limitations introduced by undocumented care at non-TriNetX affiliated organizations were recognized in few recent publications (2,23).
Limitations related to outside care may be overcome in circumstances wherein TriNetX’s natural language processing extracts facts from provider-written charts that are not recorded elsewhere in the EHR such as physician-written histories of outside organization ophthalmic care. Although natural language processing does not completely circumvent this limitation, it may provide additional information and should be considered a notable strength of the platform.
Reproducibility
Regarding reproducibility and replication of results, researchers should be aware that because TriNetX frequently updates the data available within the database, results may not be completely replicable. For analyses conducted following an initial study, it is possible that the dataset will have undergone updates, introducing new data points to the analysis, and changing outcomes or effect sizes. Additionally, researchers should be aware that, if they attempt to replicate a previous study, the present study must be designed in the same manner to achieve approximate replication as changes to cohorts, matched variables, temporal relationships, and outcomes can dramatically alter results.
Unmeasured variables
Lastly, a considerable limitation to data availability is related to an inability to measure disease severity, duration of disease, or duration of exposure to medications, all of which may impact the target outcomes (4,9,16,18,19,22). Similarly, the possibility of failure to update patient diagnosis codes in EHRs as conditions progress results in an incomplete clinical picture of study populations.
Curated data sets
Although TriNetX has limitations in coding, curated data sets provide some standardization by creating “fit-for-purpose” data sets designed for specific fields, including therapeutic areas such as cardiometabolic conditions, respiratory syncytial virus, inflammatory bowel disease, or hypertrophic cardiomyopathy. TriNetX utilizes natural language processing to extract pertinent data from clinical reports and subsequently consolidates this data using standardized clinical terminology (29). This data can include clinical information such as medications, comorbidities, or labs, and demographic data such as race and ethnicity. The data averages for the previously mentioned curated datasets are refreshed every two to four weeks (33). Because this information is regularly updated and reviewed to align with specific therapeutic areas, curated datasets may provide up-to-date, relevant data to expedite and simplify data collection and processing for clinical researchers. Furthermore, the extracted data undergoes quality assessment processes, including “data cleaning” to improve data completeness and reliability. This data cleaning may include filtering patient information using a list of required data fields and removing records that are missing required information from the dataset. For example, patient records that only include demographic information are rejected from the curated dataset (34). By isolating data from areas of interest and subsequently filtering this data, TriNetX curated data sets can provide a large volume of area-specific data that provides a degree of reliability for targeted research purposes. Although these curated datasets are not available for ophthalmology at this time, the addition of such an option to the platform may aid in mitigating limitations relation to TriNetX use in ophthalmology.
Volumetric advantages
Despite limitations related to coding and data availability, TriNetX allows high-powered analysis of outcomes, including rare outcomes, due to the volumetric advantages imparted by the platform. Currently, TriNetX accumulates over seventy billion data points of clinical observations for analysis (35). The breadth of this database creates the opportunity for a greater volume of data and a degree of accessibility for clinical researchers that may not have been previously available. Further, TriNetX ensures that all data is HIPAA-compliant and maintains an Information Security Management System, both of which alleviate ethical concerns faced by researchers in other modalities of data collection (36).
Matching
Propensity score matching between cohorts within TriNetX is inherently limited by the variables authors choose to include in their analyses. This selection process can overlook significant unconsidered confounders. Moreover, propensity matching cannot consistently eliminate baseline differences between study cohorts, introducing known confounders. The persistence of residual unmeasured confounders and residual baseline differences following propensity score matching was the second most frequently noted limitation in TriNetX ophthalmology studies (3-5,9-11,13,18-20,22,23).
An additional nuance introduced by propensity score matching that must be considered by researchers is over-exclusion. Over-matching results in large-scale exclusion of patients from the study population in an attempt to eliminate baseline differences between groups, which may decrease the external validity of TriNetX studies. Many recently published papers describe their study populations as having specific groups in proportions inconsistent with the general population with most data being obtained in the US (9,11,19,23,24). As a strength, the platform does notify researchers once they have bypassed a specified number of covariates based on the study population, however, the researcher can bypass this notification and complete the analysis.
Many of the limitations introduced by propensity score matching can be overcome by downloading the data provided by TriNetX for multivariate analysis. Although this is a solution, it takes significantly more time and computer power, making the analytics integrated into TriNetX an attractive option for rapid analysis.
Bias
The TriNetX platform is inherently subject to notable biases. TriNetX’s original purpose as an industry-academic collaboration platform means that the organizations included in the database may not be representative of the organizations accessible to the general public, limiting the external validity of conclusions generated by these studies. This is particularly relevant to ophthalmology, as individual-level patient factors can significantly influence where patients receive ophthalmic care (private vs. organizations participating in TriNetX). Furthermore, the sizes, locations, and individual characteristics of participating organizations are not readily available to users, making it difficult to ascertain the true impact of selection bias on results. Additionally, inherent variations in healthcare access and quality among the included population may not be representative of the general population, especially concerning patient access to large HCOs with vast resources (21). Despite these considerations, due to the volumetric advantages of TriNetX, it is able to provide a diverse set of data that would not otherwise be available to some researchers.
Temporal bias must also be considered, as the database is not updated in real-time (35). TriNetX reports that 80% of participating HCOs update their data at 1-, 2-, or 4-week intervals with an average data refresh frequency of one month (37). This delay in data entry may impact the perceived temporal relationship between the factors being studied when not properly addressed in a study design by accurately defining the desired temporal relationship. Temporal bias as it relates to study design can be overcome by careful design planning and use of the TriNetX study design assistant.
Another significant bias in TriNetX research is medical surveillance bias. Given that many TriNetX-affiliated organizations are large HCOs, the patient population has access to higher levels of surveillance and intervention than the general population. Moreover, individuals with more serious conditions may have increased access to and utilization of healthcare at larger organizations, leading to greater detection and reporting of ophthalmologic pathology (23).
Analytical challenges
Analytical challenges arise when using the TriNetX platform for retrospective cohort analyses. Given the high likelihood of confounding variables, missing data, and incomplete data, more advanced statistical analysis may be required to improve the quality and validity of conclusions. While more advanced analytics within TriNetX include propensity score matching and Kaplan-Meier survival curves, these methods may not be robust enough to overcome the limitations inherent to the data on the platform.
More recently, TriNetX introduced a research environment in which users can program their own analytics (3). Allowing more sophisticated analyses may serve as a promising tool to increase the strength of the studies being done on the platform and address the limitations associated with using TriNetX. However, its utility will be limited to individuals trained in, or familiar with, programing and advanced statistics.
Conclusions
In conclusion, TriNetX is a powerful research tool that has seen increased utilization in ophthalmology research in the last two years. The volumetric advantages and time efficiency imparted by TriNetX make it an attractive option for conducting observational studies. Although the platform provides researchers with access to data and analytics that may not be available through other avenues, significant caution must be exercised when designing studies to ensure their validity and quality. Despite its limitations, with careful informed use and utilization of TriNetX support systems, TriNetX is a valuable resource in ophthalmology research. As TriNetX use in ophthalmology research continues to grow, further reviews of the validity of study methodology, strengths, and limitations are warranted.
Acknowledgments
Funding: None.
Footnote
Reporting Checklist: The authors have completed the Narrative Review reporting checklist. Available at https://aes.amegroups.com/article/view/10.21037/aes-24-27/rc
Peer Review File: Available at https://aes.amegroups.com/article/view/10.21037/aes-24-27/prf
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://aes.amegroups.com/article/view/10.21037/aes-24-27/coif). A.G.L. serves as an unpaid editorial board member of Annals of Eye Science from June 2024 to December 2026. The other authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
References
- TriNetX [Internet]. [cited 2024 Aug 8]. Expand your research opportunities and enrich your data. Available online: https://trinetx.com/healthcare-organizations/
- Topaloglu U, Palchuk MB. Using a Federated Network of Real-World Data to Optimize Clinical Trials Operations. JCO Clin Cancer Inform 2018;2:1-10. [Crossref] [PubMed]
- Palchuk MB, London JW, Perez-Rey D, et al. A global federated real-world data and analytics platform for research. JAMIA Open 2023;6:ooad035. [Crossref] [PubMed]
- Dorney I, Shaia J, Kaelber DC, et al. Risk of New Retinal Vascular Occlusion After mRNA COVID-19 Vaccination Within Aggregated Electronic Health Record Data. JAMA Ophthalmol 2023;141:441-7. [Crossref] [PubMed]
- Hsia NY, Hsu AY, Wang YH, et al. The risk assessment of uveitis after COVID-19 diagnosis: A multicenter population-based study. J Med Virol 2023;95:e29188. [Crossref] [PubMed]
- Raiker R, Akosman S, Foos W, et al. Examining the Influence of COVID-19 Infection and Pandemic Restrictions on the Risk of Corneal Transplant Rejection or Failure: A Multicenter Study. Semin Ophthalmol 2023;38:777-83. [Crossref] [PubMed]
- Wai KM, Knapp A, Ludwig CA, et al. Risk of Stroke, Myocardial Infarction, and Death After Retinal Artery Occlusion. JAMA Ophthalmol 2023;141:1110-6. [Crossref] [PubMed]
- DeYoung C, Asahi MG, Rosenberg S, et al. Ophthalmology procedure trends in the United States during the COVID-19 pandemic. Int Ophthalmol 2023;43:4651-68. [Crossref] [PubMed]
- Kuo HT, Chen CY, Hsu AY, et al. Association between immune checkpoint inhibitor medication and uveitis: a population-based cohort study utilizing TriNetX database. Front Immunol 2023;14:1302293. [Crossref] [PubMed]
- Mammo DA, Wai K, Rahimy E, et al. Association of Cutaneous Keloids, Hypertrophic Scarring, and Fibrosis with Risk of Postoperative Proliferative Vitreoretinopathy. Ophthalmology 2024;131:961-6. [Crossref] [PubMed]
- Chauhan MZ, Elhusseiny AM, Kishor KS, et al. Association of Primary Open-Angle Glaucoma with Diabetic Retinopathy among Patients with Type 1 and Type 2 Diabetes: A Large Global Database Study. Ophthalmology 2024;131:827-35. [Crossref] [PubMed]
- Huang PC, Lin CC, Dana R, et al. Epidermal Growth Factor Receptor Inhibitors for Lung Cancer and the Risk of Keratitis. JAMA Ophthalmol 2024;142:140-5. [Crossref] [PubMed]
- Wai KM, Ludwig CA, Koo E, et al. Risk of Stroke, Myocardial Infarction, Deep Vein Thrombosis, Pulmonary Embolism, and Death After Retinal Vein Occlusion. Am J Ophthalmol 2024;257:129-36. [Crossref] [PubMed]
- Choudhury RA, Adducci AJ 2nd, Sarwar H, et al. Researching Eyesight Trends IN ADHD (RETINA). J Atten Disord 2024;28:236-42. [Crossref] [PubMed]
- Li JX, Wei JC, Wang YH, et al. RETINAL VASCULAR OCCLUSION AND COVID-19 DIAGNOSIS: A Multicenter Population-Based Study. Retina 2024;44:345-2. [PubMed]
- Lai JYM, Riley DR, Anson M, et al. Cardiovascular Outcomes with Intravitreal Anti-Vascular Endothelial Growth Factor Therapy in Patients with Diabetes: A Real-World Data Analysis. Diabetes Ther 2024;15:833-42. [Crossref] [PubMed]
- Tran EM, Wai KM, Kossler AL, et al. Association of Ocular Manifestations of Marfan Syndrome With Cardiovascular Complications. Am J Ophthalmol 2024;264:85-9. [Crossref] [PubMed]
- Wai KM, Mishra K, Koo E, et al. Impact of GLP-1 Agonists and SGLT-2 Inhibitors on Diabetic Retinopathy Progression: An Aggregated Electronic Health Record Data Study. Am J Ophthalmol 2024;265:39-47. [Crossref] [PubMed]
- Eleftheriadou A, Riley D, Zhao SS, et al. Risk of diabetic retinopathy and diabetic macular oedema with sodium-glucose cotransporter 2 inhibitors and glucagon-like peptide 1 receptor agonists in type 2 diabetes: a real-world data study from a global federated database. Diabetologia 2024;67:1271-82. [Crossref] [PubMed]
- Russell MW, Kumar M, Li A, et al. Incidence of ocular pathology following bariatric surgery for with morbid obesity across a large United States National Database. Eye (Lond) 2024;38:2603-9. [Crossref] [PubMed]
- Hsu AY, Wang YH, Lin CJ, et al. Assessing Uveitis Risk following Pediatric Down Syndrome Diagnosis: A TriNetX Database Study. Medicina (Kaunas) 2024;60:710. [Crossref] [PubMed]
- Chauhan MZ, Eleiwa TK, Abdelnaem S, et al. Risk of abducens nerve palsy following COVID-19 vaccination. J AAPOS 2024;28:103867. [Crossref] [PubMed]
- Gau SY, Liu PY, Chen SN, et al. Risk of Keratitis and Keratopathy in Hidradenitis Suppurativa Patients: A Global Federated Health Network Analysis. In Vivo 2024;38:1375-83. [Crossref] [PubMed]
- Hidinger I, Kong L, Ely A. Associations of strabismus surgery timing in childhood with mental health: a retrospective cohort study. J AAPOS 2024;28:103929. [Crossref] [PubMed]
- Huang HK, Lin SM, Loh CH, et al. Association Between Cataract and Risks of Osteoporosis and Fracture: A Nationwide Cohort Study. J Am Geriatr Soc 2019;67:254-60. [Crossref] [PubMed]
- Query Builder – TriNetX Help Center [Internet]. [cited 2024 Aug 8]. Available online: https://support.trinetx.com/hc/en-us/sections/115000497647-Query-Builder
- TriNetX Help Center [Internet]. 2021 [cited 2024 Aug 8]. How does TriNetX process the data from HCOs? Available online: https://support.trinetx.com/hc/en-us/articles/360004240994-How-does-TriNetX-process-the-data-from-HCOs
- Schulz S, Steffel J, Polster P, et al. Aligning an Administrative Procedure Coding System with SNOMED CT. Joint Ontology Workshops. 2019.
- TriNetX [Internet]. [cited 2024 Aug 8]. Natural Language Processing. Available online: https://trinetx.com/nlp/
- TriNetX | Penn Medicine Clinical Research | Perelman School of Medicine at the University of Pennsylvania [Internet]. [cited 2024 Aug 8]. Available online: https://www.med.upenn.edu/clinicalresearch/trinetx.html
- Wittenborn JS, Lee AY, Lundeen EA, et al. Validity of Administrative Claims and Electronic Health Registry Data From a Single Practice for Eye Health Surveillance. JAMA Ophthalmol 2023;141:534-41. [Crossref] [PubMed]
- Juniat V, Athwal S, Khandwala M. Clinical coding and data quality in oculoplastic procedures. Eye (Lond) 2019;33:1733-40. [Crossref] [PubMed]
- TriNetX [Internet]. [cited 2024 Aug 8]. How to build and license data sets from our federated EHR network. Available online: https://trinetx.com/products/real-world-datasets/
- Khan A, Bilal M, Morrow V, et al. Impact of the Coronavirus Disease 2019 Pandemic on Gastrointestinal Procedures and Cancers in the United States: A Multicenter Research Network Study. Gastroenterology 2021;160:2602-2604.e5. [Crossref] [PubMed]
- TriNetX [Internet]. [cited 2024 Aug 8]. Real-world data for the life sciences and healthcare. Available online: https://trinetx.com/
- Publication Guidelines - TriNetX [Internet]. [cited 2024 Aug 8]. Available online: https://trinetx.com/real-world-resources/publications/trinetx-publication-guidelines/
- TriNetX Help Center [Internet]. 2018 [cited 2024 Aug 8]. What is the range of dates covered by TriNetX data? Available online: https://support.trinetx.com/hc/en-us/articles/360004241114-What-is-the-range-of-dates-covered-by-TriNetX-data
Cite this article as: Hackl CM, Lee VA, Dunnigan JK, Loya A, Jaiswal S, Arogundade E, Lee AG. A narrative review of TriNetX in ophthalmology: applications, advantages, and pitfalls. Ann Eye Sci 2024;9:16.