Epidemiology

Epidemiology

About this technology

Overview Applications Our experts Contact

Our applications

Our methodological research is focused on the following topics:

  • Prediction model research methods from model development & validation to impact assessment & implementation in practice
  • Methods for development, evaluation, upscaling, and regulation of AI based health innovations
  • Methods for diagnostic test and biomarker research from technical performance to healthcare impact
  • Natural language processing and large language models for use in biomedical research and healthcare practice
  • Ecological impact of healthcare innovations and health research
  • Systematic Reviews and Meta Analysis (MA), including individual participant data MA, network MA, and diagnostic test, prognostic factor and prediction model MAs
  • Cost-effectiveness and health technology assessment of health innovations
  • Reporting guidelines and risk of bias instruments
  • Counterfactual prediction

Prediction models and AI algorithms provide estimates of an individual’s probability of the presence (diagnostic) or future occurrence (prognostic) of some outcome, based on individual’s characteristics (e.g., age, disease history, treatments). These estimates can help inform patients, their relatives, but also healthcare professionals and patients to guide treatment or lifestyle decisions, optimize work flow processes and reduce administrative burden or work stress. Traditionally, science is largely focused on the development of these prediction models, whilst providing limited evidence of their validation, generalizability or impact and cost-effectiveness of using these models on healthcare decision making, patient outcomes, healthcare processes or healthcare professionals behaviour and workloads.

We work on and provide comprehensive guidance and assistance in methods and techniques for all steps of the prediction modelling journey, including on: (i) model development: covering topics like predictor selection, assessing non-linearity, dealing with missing data, internal validation, and assessment of model performance; (ii) model validation: assessing how well the model performs in a setting not used for model development and how well it generalizes; (iii) model impact assessment: to determine its impact of the use and implementation of prediction models and AI algorithms on decision making, patient outcomes, healthcare professional workload and burden, workflow processes; (iv) cost-effectiveness or HTA analysis: using modeling studies and empirical studies (e.g., RCTs, stepped-wedge designs); (v) guidelines for assessing the trustworthiness, transparency accountability and regulation of AI algorithms and prediction models.

We developed a range of quality assessment, reporting, scientific conduct and regulatory guidelines and frameworks on this, and applied them to numerous empirical prediction models and AI innovations across all healthcare settings and domains.

Experts: Carl Moons, Ewoud Schuit, Anne de Hond, Valentijn de Jong, Kim Luijken, Hans Reitsma, Anneke Damen, Tuur Leeuwenberg, Kevin Jenniskens

Key publications:

Healthcare ecosystems throughout the world are grappling with three mutually reinforcing problems: (1) the persistent lack of personalised care, not only of treatment but also of personalised diagnosis, prognosis, monitoring, screening, and prevention of major diseases and their consequences; (2) Rising healthcare demands and costs, and an escalating demand for qualified healthcare personnel; (3) Slow, costly, and inefficient development and evaluation cycles for medical innovations, including medical technologies and pharmaceutical drugs.

AI, including machine learning, is a disruptive, rapidly emerging technology that enters all sectors, including the health sector and may address all of these challenges.

Besides experience in the empirical evaluation of the safety, effectiveness, transparency and trustworthiness of AI innovations to address the three challenges above, we are also renown for developing methodological frameworks, templates and criteria for the development, validation, impact evaluation, upscaling and regulation of safe, valid, effective, transparent and generalisable AI in healthcare. We contribute and lead numerous international AI-guidelines consortia, yielding various guidelines to enhance the development and evaluation of AI, the assessment of AI, the transparent reporting of AI research, and enhance the regulation, reimbursement and ethics of AI.

Experts: Carl Moons, Ewoud Schuit, Tuur Leeuwenberg, Anne de Hond, Valentijn de Jong, Hans Reitsma, Anneke Damen, Pauline Heus, Lotty Hooft

Key publications:

Our experts can provide guidance and assistance in the design, execution, analysis, and reporting of diagnostic test evaluation studies. This includes the full range of diagnostic test evaluation studies starting with from technical and analytical validity studies to diagnostic accuracy studies to clinical studies measuring the impact of tests on patient outcomes and costs. Our services include advice about building a portfolio of evidence for FDA or CE approval, consultation about protocols, designing studies, writing of clinical protocols and Statistical Analysis Plans, and performing statistical analyses.

Experts: Carl Moons, Kevin Jenniskens, Anneke Damen, Hans Reitsma

Key publications:

Natural language is a critical tool for communication and recording important information across all domains of health care and research. Automatic natural language processing (NLP) is methodologically challenging due to the complex structure of language but forms the basis for many important applications (from automatic note taking and health record summarization in patient care to cohort selection and variable extraction for research).

Our team works on and can provide guidance and assistance in methods and techniques for the development, evaluation and responsible application of NLP models – including large language models (LLMs) – in health care and research. Specific topics of expertise include: quality standards and the evaluation of LLM applications, automatic extraction of fine-grained information from free text in electronic health records and its consequent use in healthcare research (also for non-English), and AI-assisted systematic reviewing (e.g., for screening and data extraction).

Experts: Tuur Leeuwenberg, Anne de Hond, Anneke Damen, Carl Moons

Key publications:

Healthcare needs to reduce its environmental impact as it significantly contributes to environmental degradation. Similarly, clinical research contributes 100 megatons CO2 emissions – comparable to that of a small country – while the devastating effects of climate change worsen health outcomes. The paradox is clear: healthcare and research meant to improve health must itself become more sustainable. Yet, clinical researchers lack targeted, practical, and accessible guidance to minimize their environmental footprint, and care management decisions should consider not only effectiveness and cost-effectiveness, but also ecological sustainability.

Our team works on making healthcare and healthcare research more ecologically sustainable by highlighting its importance via opinion pieces and literature reviews, by developing decision support tools to consider environmental impact in care management decisions, and by developing guidance for researchers on how they can lower the ecological footprint of their research while maintaining its quality. Two recently funded research projects, one by ZonMw and one by NWO, will further strengthen this line of research.

Experts: Ewoud Schuit, Pauline Heus

Key publications:

Systematic reviews and meta-analyses provide the most comprehensive synthesis of evidence to address clinical and public health questions. These approaches enable rigorous evaluation of the effectiveness, safety, and cost-effectiveness of interventions, as well as the development and evaluation of diagnostic and prognostic models.

We have specific expertise in individual participant data (IPD) meta-analysis, which allows harmonisation of measurements and analytic approaches across studies. This enables detailed investigation of heterogeneity and effect modification using all available data, rather than relying on published summary results that may be affected by selective reporting or publication bias. In this setting, diagnostic and prognostic models can also be refined to improve generalisability and reduce variation in performance across populations.

When multiple interventions are available but have not all been directly compared in head-to-head studies, we apply network meta-analysis (NMA) to integrate direct and indirect evidence. This allows simultaneous comparison and ranking of competing interventions. Similarly, NMA can be used when multiple diagnostic or prognostic models have been evaluated across different studies, and head-to-head evidence is limited.

When evidence from randomised controlled trials is limited or unavailable, we can apply cross-design methods, which combine evidence from both randomised and non-randomised studies, while explicitly accounting for differences in study quality and risk of bias.

Our team focuses on the development, evaluation, and application of state-of-the-art methodology for systematic reviews and meta-analysis. We can provide guidance to design evidence syntheses, and on the appropriate and transparent use of advanced statistical methods.

Experts: Valentijn de Jong, Anneke Damen, Hans Reitsma, Carl Moons, Kevin Jenniskens, Pauline Heus, Kim Luijken, Lotty Hooft

Key publications:

Reporting guidelines are increasingly used and mandated in all types of medical research. They promote transparent and complete reporting which generate multiple positive consequences, including enhancing reproducibility of research, reducing research waste, facilitating critical appraisal, and supporting evidence based healthcare. Risk of bias instruments assess the likelihood that the design, conduct, analysis, or reporting of a study has introduced systematic errors in the reported results (bias). These instruments play a critical in determining the trustworthiness of the findings of a study. The risk of bias assessment is an essential step when formulating clinical guidelines and when performing a systematic review.

Our team is involved in the development and updating of multiple reporting guidelines and risk of bias instruments. We can provide expertise and guidance in projects where there are challenges in applying existing tools or in initiatives that aim to generate a new instrument or an extension.

Experts: Anneke Damen, Pauline Heus, Lotty Hooft, Carl Moons, Hans Reitsma

Key publications:

Prediction models are increasingly used to guide treatment decisions. However, many are developed using observational data in which some patients have already received the treatment that the model is meant to inform. This can introduce bias and lead to misleading conclusions if not carefully addressed. Understanding what a prediction model can and cannot be used for is essential to avoid inappropriate decisions and potential harm to patients.

We provide:

  • Consultation decision-making: Assessing whether a prediction model is appropriate for guiding treatment decisions.
  • Consultation model development & evaluation: Clarifying the prediction target and aligning models with the underlying clinical question; Advice on data requirements and methodological approaches that integrate prediction modeling with causal inference principles.
  • Collaboration on research projects or education on the responsible development and use of prediction models, with a strong focus on causal reasoning.

Experts: Kim Luijken, Ewoud Schuit, Tuur Leeuwenberg

Key publications: