International Guideline for Routinely Collected Health Data

About the Guideline

Data such as electronic health records, patient registries, and administrative healthcare data are in general not collected with research purposes in mind. This creates challenges related to data quality, representativeness, timing of measurements and interventions, confounding and the multiplicity of possible analysis strategies. The guideline discusses how these challenges can lead to unreliable results. It also notes that AI and other advanced methods can yield misleading results and reinforce existing biases if the underlying data are of poor quality or if they are implemented without clinical and methodological expertise and rigorous internal and external validation.

Key Recommendations

To improve the reliability of research based on routinely collected data:

• Assess data quality before analysis to ensure that routinely collected data are suitable for the research question, ideally by assessing data quality in a validation study in which data are collected prospectively to characterize and quantify measurement error and missing data patterns
• Promote transparency and reproducibility by registering and clearly documenting and reporting analytical decisions.
• Define and report timing carefully to align interventions, eligibility , and outcomes to reduce self-inflicted biases.
• Address missing data and measurement error. Consider representativeness, treatment allocation, and diagnostic tests not being random.
• Validate AI and predictive models rigorously by using robust validation and combining AI methods with domain expertise.
• Prioritize high-quality evidence. When data quality is insufficient, prospective data collection or even not studying a particular research question may be preferable, as results may otherwise be misleading and overinterpreted.

The guideline aims to strengthen confidence in findings derived from routinely collected health data and support more robust evidence-based medicine.