> ## Content Index
> Fetch the complete content index at: https://www.healthdatacon.io/llms.txt
> Use this file to discover other available public pages before exploring further.

# Real-world evidence needs quality by design before analysis begins
- URL: https://www.healthdatacon.io/articles/real-world-evidence-needs-quality-by-design-before-analysis-begins/
- Published: 2026-11-15T13:05:00.000Z
- Updated: 2026-11-19T04:33:21.000Z
- Description: Applying clinical trial principles to real-world research can strengthen data reliability, establish fitness for purpose and produce credible evidence for clinical and regulatory decision-making.
- Author: Jess Paulus
- Tags: Informatics, Americas, FACHDM

For years, healthcare and life sciences leaders asked whether electronic health record data could support research and regulatory decisions. That question is no longer theoretical. The more important question now is whether a particular real-world dataset is sufficiently relevant, reliable and complete for the decision a study is meant to inform.

The policy history matters because two different federal developments are often blended together. The [HITECH Act](https://healthit.gov/data/data-briefs/non-federal-acute-care-hospital-electronic-health-record-adoption-2008-2024/?ref=healthdatacon.io) of 2009 drove major investment in certified EHR adoption and meaningful use, expanding the digital clinical data available for secondary analysis. The [21st Century Cures Act](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-use-real-world-data-and-real-world-evidence-support-regulatory-decision-making-drug?ref=healthdatacon.io) of 2016 then directed FDA to establish a program evaluating the potential use of real-world evidence in regulatory decision-making for drugs and biologics.

That evolution has changed the evidentiary standard. The fact that data originated in routine care does not make them inherently suitable or unsuitable for research. Fitness for purpose depends on the question, the population, the variables required, how those variables were generated and transformed, and whether the resulting endpoints and covariates are reliable enough to support the intended inference.

## The clinical-trial lesson is prospective quality design

The useful analogy to a clinical trial is not that an observational study should imitate randomization or pretend routine-care data were collected under a trial protocol. It is that important quality requirements should be identified before the analysis begins rather than retrofitted after a result has already been produced.

That principle is well established in clinical research. ICH [E8(R1)](https://database.ich.org/sites/default/files/ICH%5FE8-R1%5FGuideline%5FStep4%5F2021%5F1006.pdf?ref=healthdatacon.io) describes quality by design as a prospective approach in which critical-to-quality factors are built into study design, planning and conduct. The guideline is explicit that retrospective review and monitoring remain useful but are not sufficient substitutes for designing quality into the study from the outset.

Real-world studies intended to influence clinical or regulatory decisions need the same discipline. Before the analysis is run, investigators should know which variables are essential, how much missingness is acceptable, whether the source population represents the intended population, how exposure and outcomes will be ascertained, and what level of measurement error would make the study unreliable. Those thresholds do not have to be identical for every study because the acceptable level of uncertainty depends on the question being asked.

This is also where platform-level quality and study-level fitness for purpose need to be distinguished. Organizations that repeatedly use the same data asset can and should build persistent controls for provenance, transformations, coding consistency, missingness and anomaly detection. But no platform can declare itself universally 'fit for purpose.' The final judgment still belongs to the specific research question and protocol.

## FDA and EMA are making up-front characterization more explicit

FDA's Oncology Center of Excellence provides one example through the [QCARD initiative](https://www.fda.gov/about-fda/oncology-center-excellence/oncology-quality-characterization-and-assessment-real-world-data-qcard-initiative?ref=healthdatacon.io), developed with the Reagan-Udall Foundation. QCARD is not a regulatory certification of data quality and it is not a fixed eight-item scorecard. Instead, FDA publishes commonly requested design and data-source elements that can help sponsors characterize an early oncology RWD proposal before analysis.

Those elements include the study question, data-source rationale, study period, intended population, exposure and comparator definitions, endpoints, key covariates, planned analyses, data availability, provenance, missingness and the data-quality plan. FDA also states that formal protocols and statistical analysis plans should be finalized before study initiation and analysis, reinforcing the idea that characterization should precede the result.

Europe is moving in the same general direction, although through a different framework. EMA's [Data Quality Framework](https://www.ema.europa.eu/en/about-us/how-we-work/data-regulation-big-data-other-sources/data-quality-framework-medicines-regulation?ref=healthdatacon.io) evaluates data quality through five dimensions: reliability, extensiveness, coherence, timeliness and relevance. In March 2026, EMA published a real-world-data application of that framework that makes the question-specific nature of fitness for purpose especially clear.

The FDA and EMA approaches should not be treated as identical or fully harmonized. They differ in scope and structure. What they share is more important for research operations: sponsors need to understand how the data were generated, whether the necessary information exists, how transformations are governed, and whether the dataset is appropriate for the particular scientific and regulatory question.

## What platform-level quality looks like in practice

Turning those principles into decision-grade evidence requires infrastructure that operates before any one study starts. Oncology is a useful stress test because clinically important variables are distributed across structured EHR fields, narrative notes, laboratory and pathology systems, claims, mortality sources and other linked datasets. No single source is guaranteed to contain the complete patient story.

A demographic field or administered drug may be captured reliably in structured data, while histology, progression status, oral medication discontinuation or the reason a therapy stopped may require chart abstraction or natural-language processing. The appropriate response is not to assume that one source is authoritative for everything; it is to define the source hierarchy and transformation rules for each variable the research program expects to use.

Continuous quality monitoring is equally important. Automated checks can identify impossible dates, missing fields, duplicated records, implausible treatment sequences and changes in data distributions before those problems propagate across multiple studies. Finding anomalies during routine monitoring is not evidence that a dataset is unusable. A functioning quality system should find discrepancies, document them and create a repeatable path for correction or qualification.

This is where enterprise-level governance can reduce the amount of quality work that has to be reinvented for every protocol. A platform can maintain data dictionaries, lineage, common transformation logic, source-specific validation, audit trails and monitoring thresholds as reusable infrastructure. A study team can then focus its additional effort on the variables and biases that are critical to the question at hand.

## The oncology example needs both scale and disclosure

A practical example comes from the author's own organization, Ontada, which provides oncology data and technology services and works with The US Oncology Network. That relationship is material because the article draws on the network and its EHR environment as an implementation example. The value of the example is not the size of a commercial platform by itself; it is the way a large, repeatedly used data environment can support persistent quality controls that are then tested again at the study level.

Current network figures show the scale has changed from earlier descriptions. The US Oncology Network now reports [3,300+ providers](https://usoncology.com/our-company/?ref=healthdatacon.io), more than 750 sites of care and more than 2 million patients treated annually. Ontada's iKnowMed EHR is used by more than 2,800 oncology providers, but those figures should be understood as organizational scale, not evidence that the underlying data are automatically suitable for every research question.

The more consequential quality question is how the platform combines structured data with chart abstraction, NLP-derived variables and external sources while preserving provenance. Each enrichment step can improve completeness, but it also creates another transformation that needs documentation, monitoring and validation.

## Endpoint validation is where fitness for purpose becomes empirical

Some data-quality questions can be answered through documentation and process controls; others require comparison against an external reference. Mortality is a good example because a patient can disappear from an oncology EHR after transferring care, entering hospice or dying outside the health system. An EHR-derived mortality endpoint that appears plausible may still miss deaths and overestimate survival.

Ontada researchers have examined this problem by comparing multiple real-world mortality sources with the National Death Index in high-mortality cancers, including metastatic pancreatic cancer. An [ISPOR analysis](https://www.ispor.org/heor-resources/presentations-database/presentation/intl2024-3900/137785?ref=healthdatacon.io) used EHR data, financial/claims information, government mortality files and a commercial mortality source, with NDI serving as an external benchmark. The point is not that NDI should be linked for every study; it is that important endpoints should be validated against credible reference standards when the scientific risk justifies it.

That validation mindset is broader than mortality. Researchers may need to validate NLP-derived progression, biomarker status, treatment exposure, cause-specific outcomes or other variables whose accuracy cannot be inferred from completeness alone. The appropriate validation design will vary, but the principle is the same: important endpoints should earn trust through evidence rather than through face validity.

## What comes next for real-world evidence

The first likely evolution is that more quality control will move into the data-generating infrastructure. Reusable provenance, automated edit checks, transformation monitoring and source-level validation can make every downstream study more efficient, provided investigators still perform question-specific fitness-for-purpose assessment. The goal is not to eliminate study-level quality work but to stop rediscovering the same platform defects one protocol at a time.

The second evolution is greater regulatory alignment around the language of fitness for purpose. FDA and EMA will continue to use different tools and legal frameworks, but both are asking sponsors to make the relationship among the research question, the data source and the quality evidence more transparent. Organizations that understand both frameworks will be better positioned to build evidence packages that can travel across regulatory settings without claiming a harmonization that does not yet exist.

The third evolution is more reusable endpoint validation. Instead of treating every mortality, progression or exposure algorithm as a one-off analytic choice, mature data platforms can maintain validation evidence that is refreshed as sources and algorithms change. That approach creates an evidence trail for the measurement itself, while still requiring investigators to decide whether the validated endpoint is suitable for the particular study population and use case.

The larger lesson is that real-world evidence does not become decision-grade because a dataset is large, longitudinal or technologically sophisticated. It becomes credible when the research question, data provenance, quality thresholds, endpoint definitions and validation strategy are specified before the answer is known. That is the part of clinical-trial discipline worth borrowing: quality is most persuasive when it is designed into the evidence process rather than assembled after the result.