Course Introduction
The Advanced Quasi-Experimental Methods for Impact Evaluation Training Course provides a rigorous framework for designing, implementing, analyzing, and interpreting evaluations when randomized controlled trials are impractical, unethical, too costly, or unavailable. Participants will develop advanced knowledge of causal inference, counterfactual reasoning, identification strategies, and empirical designs that use naturally occurring variation, policy changes, institutional rules, geographic boundaries, timing differences, and administrative processes to estimate program and policy impacts.
The course develops practical expertise in major quasi-experimental designs, including difference-in-differences, regression discontinuity, instrumental variables, interrupted time series, synthetic control methods, matching approaches, event-study designs, and comparative case-based strategies. Participants will examine the assumptions underlying each design, understand when identification is credible, recognize common threats to validity, and learn how design choices influence causal interpretation. Emphasis is placed on connecting research questions and institutional settings with appropriate identification strategies.
Advanced treatment is given to modern developments in difference-in-differences and related panel-data methods, including staggered treatment adoption, heterogeneous treatment effects, dynamic effects, treatment reversals, anticipation, and alternative comparison-group construction. Participants will learn how conventional two-way fixed-effects approaches can behave differently under heterogeneous treatment effects and how newer estimators and event-study strategies can improve interpretation. The course also addresses practical issues involving repeated observations, clustered errors, treatment timing, parallel-trends assessment, and sensitivity to specification choices.
Participants will also explore advanced regression discontinuity, instrumental-variable, and synthetic-control applications. Topics include bandwidth selection, manipulation testing, continuity assumptions, local treatment effects, weak instruments, exclusion restrictions, heterogeneous effects, placebo tests, donor-pool construction, pre-treatment fit, and sensitivity analysis. Practical exercises will demonstrate how apparently sophisticated models can produce misleading conclusions when the underlying identification assumptions are weak or when implementation decisions are not transparently documented.
The course places strong emphasis on research credibility, robustness, reproducibility, and transparent impact-evaluation reporting. Participants will learn to conduct falsification tests, placebo analyses, specification checks, sensitivity analyses, balance assessments, pre-trend evaluations, alternative-window analyses, and robustness exercises. Attention is also given to missing data, attrition, spillovers, interference, measurement error, treatment contamination, multiple hypothesis testing, external validity, and the interpretation of statistical uncertainty in complex evaluation settings.
By the end of the course, participants will be equipped to develop defensible quasi-experimental evaluation strategies for public programs, development interventions, health initiatives, education reforms, social protection programs, regulatory changes, organizational interventions, and other real-world policies. Emerging issues such as machine-learning-assisted causal inference, heterogeneous treatment effects, high-frequency administrative data, causal forests, synthetic difference-in-differences, debiased estimation, reproducible workflows, and transparent evidence standards will be integrated into the learning experience.
Duration
10 days
Who Should Attend
-
Impact evaluation specialists responsible for designing or reviewing rigorous program evaluations.
-
Economists and applied researchers conducting causal analysis using observational or administrative data.
-
Monitoring and evaluation professionals seeking advanced quasi-experimental evaluation capabilities.
-
Policy analysts assessing the effects of government programs, reforms, regulations, and public investments.
-
Development practitioners evaluating interventions where randomized experiments are unavailable or impractical.
-
Data scientists and statisticians working on causal inference and policy-impact estimation.
-
Researchers using longitudinal, panel, administrative, survey, or repeated cross-sectional datasets.
-
Public-sector analysts responsible for evidence-based policy assessment and performance measurement.
-
Academics and research professionals developing advanced empirical evaluation studies.
-
Consultants advising organizations on evaluation design, causal identification, and impact measurement.
-
Program managers who need to interpret and communicate rigorous impact-evaluation evidence.
-
Professionals preparing evaluation reports, research papers, technical studies, or policy evidence products.
Course Objectives
-
Develop advanced understanding of quasi-experimental identification strategies and their application to real-world impact-evaluation problems where randomized assignment is unavailable.
-
Assess causal assumptions systematically and determine whether a proposed quasi-experimental design provides a credible basis for estimating treatment or policy effects.
-
Apply difference-in-differences methods to panel and repeated cross-sectional data while addressing treatment timing, heterogeneous effects, and dynamic impacts.
-
Design and interpret regression discontinuity studies using appropriate identification assumptions, bandwidth strategies, manipulation tests, and local treatment-effect interpretations.
-
Apply instrumental-variable methods to situations involving endogenous treatment assignment, carefully evaluating instrument relevance, exclusion restrictions, and local causal interpretation.
-
Implement synthetic control and related comparative-intervention approaches for estimating policy impacts using carefully constructed counterfactual trajectories.
-
Evaluate matching, weighting, and propensity-score-based approaches while understanding their assumptions, limitations, balance requirements, and relationship to causal identification.
-
Diagnose threats to internal validity including selection bias, time-varying confounding, anticipation, spillovers, attrition, measurement error, and treatment contamination.
-
Conduct advanced robustness, sensitivity, falsification, placebo, and specification analyses to determine how dependent findings are on modeling and identification choices.
-
Integrate longitudinal and administrative data structures into quasi-experimental designs while appropriately handling clustering, serial correlation, missingness, and unbalanced observations.
-
Communicate quasi-experimental findings transparently by distinguishing statistical associations, identified causal effects, assumptions, limitations, and broader policy implications.
-
Develop reproducible and technically defensible impact-evaluation workflows incorporating modern causal-inference methods, computational tools, transparent reporting, and emerging analytical approaches.
Comprehensive Course Outline
Module 1: Foundations of Advanced Quasi-Experimental Impact Evaluation
-
Counterfactual reasoning, potential outcomes, causal estimands, treatment effects, and the logic of identification in non-randomized evaluation settings.
-
Differences between experimental, quasi-experimental, observational, and descriptive designs and implications for credible causal interpretation.
-
Sources of naturally occurring variation including policy thresholds, institutional rules, geographic boundaries, timing changes, and administrative eligibility criteria.
-
Translating substantive evaluation questions into estimands, identification strategies, observable data requirements, and defensible empirical designs.
Module 2: Causal Identification, Counterfactuals, and Threats to Validity
-
Formal identification concepts including exchangeability, consistency, positivity, stable treatment assumptions, and design-specific causal requirements.
-
Selection bias, confounding, simultaneity, reverse causality, measurement error, and omitted-variable problems in observational impact evaluation.
-
Internal, external, construct, and statistical conclusion validity and how each dimension affects interpretation of evaluation evidence.
-
Building causal diagrams and structured identification arguments to clarify assumptions, pathways, confounders, mediators, colliders, and potential sources of bias.
Module 3: Difference-in-Differences Design and Implementation
-
Foundations of two-period and multi-period difference-in-differences designs for estimating impacts from changes across treated and comparison groups.
-
Parallel-trends assumptions, graphical diagnostics, pre-treatment comparisons, placebo periods, and strategies for assessing the credibility of counterfactual trends.
-
Panel versus repeated cross-sectional difference-in-differences designs, treatment-group construction, clustering, serial correlation, and inference considerations.
-
Practical implementation of specification checks, alternative comparison groups, time windows, covariates, and functional forms in difference-in-differences studies.
Module 4: Advanced Difference-in-Differences and Event-Study Methods
-
Staggered treatment adoption, heterogeneous treatment effects, dynamic impacts, and limitations of conventional two-way fixed-effects estimators.
-
Event-study specifications for examining treatment effects over time, treatment anticipation, delayed responses, and post-intervention effect persistence.
-
Modern estimators for staggered adoption settings, cohort-specific effects, group-time average treatment effects, and alternative aggregation strategies.
-
Interpretation of treatment-effect heterogeneity, treatment reversals, negative weighting concerns, and sensitivity to treatment-timing assumptions.
Module 5: Regression Discontinuity Designs
-
Sharp and fuzzy regression discontinuity designs based on known assignment thresholds and locally comparable observations.
-
Continuity assumptions, cutoff selection, bandwidth determination, polynomial specifications, local regression, and appropriate inference procedures.
-
Detecting manipulation and sorting around thresholds using density tests, covariate continuity checks, graphical diagnostics, and institutional knowledge.
-
Interpreting local treatment effects and assessing their relevance when program impacts may differ substantially away from the cutoff.
Module 6: Instrumental Variables and Natural Experiments
-
Instrumental-variable identification using exogenous variation and the conditions required for credible causal estimation.
-
Instrument relevance, exclusion restrictions, monotonicity, compliance behavior, and interpretation of local average treatment effects.
-
Weak instruments, overidentification considerations, first-stage diagnostics, robust inference, and sensitivity to alternative instrument specifications.
-
Natural experiments based on institutional rules, eligibility changes, lotteries, administrative processes, geographic variation, and externally determined shocks.
Module 7: Matching, Weighting, and Comparative Observational Designs
-
Propensity-score matching, covariate matching, inverse probability weighting, entropy balancing, and related approaches to constructing comparable groups.
-
Assessing covariate balance, overlap, positivity, common support, treatment-model specification, and sensitivity to unmeasured confounding.
-
Combining matching or weighting with regression adjustment, panel methods, difference-in-differences, and doubly robust estimation strategies.
-
Understanding the distinction between improving observed comparability and establishing causal identification when important unobserved differences remain.
Module 8: Synthetic Control and Comparative Case Methods
-
Synthetic control methodology for constructing weighted combinations of comparison units to approximate untreated counterfactual trajectories.
-
Donor-pool selection, predictor selection, pre-treatment fit, intervention timing, and interpretation of post-intervention divergence.
-
Placebo and permutation-based inference, in-space and in-time placebo analyses, and assessment of sensitivity to donor-pool composition.
-
Extensions including augmented synthetic control, synthetic difference-in-differences, generalized synthetic controls, and multiple treated units.
Module 9: Longitudinal, Panel, and Repeated-Cross-Section Evaluation
-
Structuring longitudinal and panel datasets for causal evaluation while accounting for individual, organizational, geographic, and temporal dependence.
-
Fixed effects, random effects, unit-specific trends, time effects, clustering strategies, and implications for identification in panel designs.
-
Repeated cross-sectional data, changing sample composition, survey weights, population-level interpretation, and treatment exposure measurement.
-
Handling unbalanced panels, irregular observation periods, treatment switching, entry and exit, and varying observation intensity in impact studies.
Module 10: Advanced Confounding, Spillovers, and Interference
-
Time-varying confounding and approaches for evaluating interventions where treatment assignment and covariates evolve simultaneously.
-
Treatment spillovers, geographic interference, network effects, contamination, and violations of assumptions concerning independent treatment exposure.
-
Anticipation effects, delayed implementation, partial compliance, treatment switching, and other complications affecting treatment-effect interpretation.
-
Sensitivity analysis for hidden bias, alternative exposure definitions, negative controls, and design-based strategies for strengthening causal credibility.
Module 11: Robustness, Falsification, and Sensitivity Analysis
-
Placebo outcomes, placebo interventions, pre-treatment tests, negative-control strategies, and falsification exercises for challenging causal interpretations.
-
Specification-curve thinking, alternative functional forms, bandwidths, time windows, comparison groups, covariate sets, and treatment definitions.
-
Sensitivity analyses for omitted-variable bias, unmeasured confounding, influential observations, treatment misclassification, and measurement uncertainty.
-
Transparent distinction between confirmatory and exploratory analyses and appropriate interpretation of robustness when multiple specifications are examined.
Module 12: Missing Data, Attrition, Measurement Error, and Data Quality
-
Missing-data mechanisms, attrition processes, incomplete treatment records, and implications for identification and statistical inference.
-
Multiple imputation, inverse probability weighting, complete-case approaches, and sensitivity analysis for missing outcomes and covariates.
-
Measurement error in treatment, outcomes, timing, eligibility, and covariates and its implications for attenuation, misclassification, and causal bias.
-
Administrative data linkage, record inconsistencies, duplicate observations, changing definitions, coding errors, and reproducible data-quality assessment.
Module 13: Advanced Heterogeneous Treatment Effects and Causal Machine Learning
-
Treatment-effect heterogeneity across demographic, geographic, institutional, baseline-risk, and exposure-intensity subgroups.
-
Causal forests, generalized random forests, meta-learners, heterogeneous-effect estimation, and appropriate interpretation of machine-learning-assisted causal analysis.
-
Double machine learning, orthogonalization, nuisance-function estimation, cross-fitting, and high-dimensional confounding adjustment.
-
Avoiding overfitting, data leakage, selective subgroup analysis, multiple testing, and misleading individualized treatment-effect interpretations.
Module 14: Inference, Model Diagnostics, and Reproducible Evaluation
-
Clustered, heteroskedasticity-robust, randomization-based, bootstrap, and other inference approaches appropriate to quasi-experimental evaluation designs.
-
Model diagnostics, influential observations, residual dependence, specification problems, leverage, functional-form assumptions, and sensitivity to analytical choices.
-
Reproducible workflows using version-controlled code, documented datasets, computational environments, analysis scripts, and transparent research protocols.
-
Pre-analysis plans, registered reports, replication materials, reporting standards, and strategies for reducing analytical flexibility and improving research credibility.
Module 15: Impact Evaluation Applications and Evidence Interpretation
-
Applying quasi-experimental methods to education, health, labor markets, social protection, infrastructure, environmental policy, governance, and development programs.
-
Translating statistical treatment effects into meaningful program outcomes, implementation implications, cost-effectiveness considerations, and decision-relevant evidence.
-
Interpreting statistical significance, confidence intervals, effect sizes, practical significance, heterogeneous effects, and uncertainty without overstating causal conclusions.
-
Assessing generalizability, transportability, implementation context, policy scale-up, and differences between estimated local effects and broader population impacts.
Module 16: Integrated Evaluation Projects and Emerging Directions
-
Designing an end-to-end quasi-experimental evaluation from research question and estimand through data preparation, identification, estimation, diagnostics, and reporting.
-
Integrating modern methods such as synthetic difference-in-differences, advanced event studies, doubly robust estimators, machine learning, and high-dimensional administrative data.
-
Emerging challenges involving real-time policy evaluation, automated analytical workflows, AI-assisted causal discovery, privacy-preserving data linkage, and increasingly complex observational datasets.
-
Preparing transparent technical reports and evidence products that document assumptions, methods, robustness checks, limitations, causal claims, and reproducibility requirements.
Training Approach
This course will be delivered by our skilled trainers who have vast knowledge and experience as expert professionals in the fields. The course is taught in English and through a mix of theory, practical activities, group discussion and case studies. Course manuals and additional training materials will be provided to the participants upon completion of the training.
Tailor-Made Course
This course can also be tailor-made to meet organization requirement. For further inquiries, please contact us on: Email: training@upskilldevelopment.com Tel: +254 721 331 808
Training Venue
The training will be held at our Upskill Training Centre. We also offer training for a group (at a discount of 10% to 50%) at requested location all over the world. The Onsite course fee covers the course tuition, training materials, two break refreshments, buffet lunch, airport transfers, Upskill gift package, and guided tour.
Visa application, travel expenses, dinners, accommodation, insurance, and other personal expenses are catered by the participant
Certification
Participants will be issued with Upskill certificate upon completion of this course.
Airport Pickup and Accommodation
Airport pickup and accommodation is arranged upon request. For booking contact our Training Coordinator through Email: training@upskilldevelopment.com, +254 721 331 808
Terms of Payment:
Unless otherwise agreed between the two parties’ payment of the course fee should be done 3 working days before commencement of the training so as to enable us to prepare better.
