Course Introduction
Counterfactual reasoning is a fundamental approach to understanding whether observed changes in individuals, households, institutions, or communities can reasonably be attributed to a development intervention. In many programme settings, randomized controlled trials are impractical, prohibitively expensive, ethically inappropriate, or inconsistent with operational realities. Evaluators must therefore use nonexperimental evidence and defensible analytical strategies to estimate what might have happened in the absence of an intervention. This course equips participants with the conceptual foundations and practical methods required to formulate counterfactual questions, identify suitable comparison conditions, assess alternative explanations, and draw appropriately qualified conclusions about programme contribution and impact.
Development interventions operate in dynamic environments where observed outcomes are influenced by multiple factors, including economic trends, government policies, demographic changes, environmental conditions, institutional reforms, and the actions of other development partners. A programme may coincide with improvements that would have occurred anyway, while an intervention with genuine benefits may appear ineffective because of adverse external conditions. Counterfactual reasoning helps evaluators distinguish programme-related changes from these competing influences. Participants will learn how to define treatment and comparison conditions, understand the fundamental problem of unobserved counterfactual outcomes, and examine the assumptions needed to support credible causal conclusions using observational data and nonexperimental evaluation designs.
The course explores the principal sources of bias that can undermine nonexperimental programme assessments. Participants will examine selection bias, confounding, reverse causality, regression to the mean, history effects, maturation, measurement error, attrition, and differences between intervention and comparison groups. Particular attention will be given to understanding how programme participation is determined, why participants may differ systematically from nonparticipants, and how these differences can distort estimated effects. Through structured analytical examples, participants will learn to develop causal diagrams, identify relevant confounders, assess data limitations, and determine whether the available evidence can support a credible comparison. The emphasis is on transparent reasoning rather than applying statistical techniques without understanding their underlying assumptions.
A major component of the training focuses on nonexperimental designs and analytical methods that can strengthen counterfactual assessment. Participants will explore matched comparison groups, propensity score methods, regression adjustment, interrupted time-series analysis, difference-in-differences, synthetic control approaches, regression discontinuity designs where applicable, and related quasi-experimental strategies. They will learn how to select methods according to programme characteristics, available data, intervention timing, eligibility rules, comparison opportunities, and plausible assumptions. The course emphasizes the importance of checking pre-intervention comparability, investigating parallel trends where relevant, assessing overlap and balance, testing sensitivity to analytical choices, and distinguishing statistical precision from the credibility of the causal design.
Counterfactual reasoning also requires evaluators to combine statistical analysis with contextual knowledge and a careful examination of alternative explanations. Participants will learn how qualitative evidence, programme records, implementation histories, stakeholder interviews, and contextual analysis can help interpret estimated effects and identify mechanisms through which change may have occurred. The course examines how mixed-methods approaches can explain unexpected findings, reveal implementation differences, and assess whether the assumptions behind a comparison remain plausible. Ethical considerations, data protection, transparency, and responsible communication are integrated throughout the training to ensure that causal claims are proportionate to the strength of the evidence and do not mislead programme stakeholders or decision-makers.
By the end of the course, participants will be able to design and critically assess counterfactual strategies for nonexperimental programme evaluations. They will gain practical experience developing causal questions, selecting comparison groups, reviewing observational datasets, identifying threats to validity, interpreting effect estimates, and documenting methodological limitations. Participants will also learn to communicate findings to programme managers, donors, policymakers, and implementing partners in ways that distinguish observed associations from credible causal evidence. These competencies support better-informed funding decisions, more rigorous programme learning, stronger accountability, and improved understanding of whether development interventions are likely to have produced meaningful changes beyond what would otherwise have occurred.
Duration
5 days
Who Should Attend
-
Monitoring and evaluation specialists responsible for assessing programme effectiveness and development outcomes.
-
Impact evaluation practitioners conducting nonexperimental and quasi-experimental programme assessments.
-
Development researchers investigating the effects of social, economic, and institutional interventions.
-
Programme managers who need credible evidence about whether interventions contribute to observed changes.
-
Economists and statisticians working with observational data and causal inference methods.
-
Monitoring, evaluation, accountability, and learning professionals strengthening evidence-based programme decisions.
-
Government policy analysts evaluating public policies and service-delivery reforms.
-
NGO and civil society evaluation teams assessing community development and social protection interventions.
-
Donor representatives and grant managers reviewing programme impact evidence and funding performance.
-
Research consultants designing evaluation strategies when random assignment is unavailable or unsuitable.
-
Public health evaluation professionals assessing nonrandomized health programmes and service interventions.
-
Education specialists evaluating learning initiatives, student support programmes, and institutional reforms.
-
Agricultural and livelihoods programme officers assessing intervention effects on household welfare and productivity.
-
Data analysts and research officers preparing observational datasets for programme evaluation.
-
Academic staff and postgraduate researchers studying impact evaluation, programme effectiveness, and causal inference.
Course Objectives
-
Explain the principles of counterfactual reasoning and causal inference, including the fundamental problem of causal inference and the role of potential outcomes in programme assessment.
-
Formulate clear evaluation questions and define treatment conditions, target populations, outcome measures, comparison conditions, and relevant assessment periods.
-
Identify selection bias, confounding, omitted-variable bias, reverse causality, attrition, and other threats to validity in nonexperimental programme assessments.
-
Develop causal diagrams and analytical frameworks to distinguish confounders, mediators, moderators, and other variables relevant to defensible causal estimation.
-
Select suitable nonexperimental and quasi-experimental evaluation designs based on programme characteristics, intervention timing, available data, and identifying assumptions.
-
Apply appropriate comparison-group strategies, matching approaches, regression adjustment, and other analytical techniques to improve comparability between intervention and comparison observations.
-
Assess key assumptions behind difference-in-differences, interrupted time series, synthetic control, and regression discontinuity approaches where relevant to programme design.
-
Conduct diagnostic checks, balance assessments, pre-trend analysis, robustness checks, and sensitivity analysis to examine the credibility of estimated programme effects.
-
Integrate qualitative evidence, implementation information, and contextual analysis to investigate alternative explanations and interpret nonexperimental findings responsibly.
-
Prepare transparent evaluation reports that communicate estimated effects, methodological assumptions, uncertainty, limitations, and implications for programme management and policy decisions.
Comprehensive Course Outline
Module 1: Foundations of Counterfactual Reasoning
-
Concepts of causality, counterfactual outcomes, treatment effects, and causal attribution in development programme evaluation.
-
Potential outcomes framework, average treatment effects, treatment-on-the-treated effects, and differences between individual and population-level effects.
-
Distinctions between experimental, quasi-experimental, and observational evaluation designs and their respective inferential strengths.
-
Practical applications of counterfactual reasoning in social protection, education, health, livelihoods, governance, and public service programmes.
Module 2: Defining the Evaluation Question and Causal Model
-
Translating programme objectives into clear causal questions, estimands, outcome indicators, target populations, and assessment timeframes.
-
Defining intervention exposure, treatment eligibility, participation status, comparison conditions, and relevant units of analysis.
-
Developing directed acyclic graphs and causal frameworks to identify confounding variables, mediators, moderators, and potential sources of bias.
-
Establishing analytical protocols that align programme theory, data availability, identification strategies, and intended evaluation conclusions.
Module 3: Threats to Validity in Nonexperimental Assessments
-
Understanding selection bias arising from voluntary participation, eligibility criteria, targeting decisions, and differences in programme access.
-
Identifying confounding, omitted-variable bias, reverse causality, simultaneity, and unobserved differences between comparison groups.
-
Assessing history effects, maturation, regression to the mean, spillovers, contamination, attrition, and changes in outcome measurement.
-
Evaluating internal validity, external validity, construct validity, and statistical conclusion validity in observational programme research.
Module 4: Comparison Groups, Matching, and Regression Adjustment
-
Selecting credible comparison groups using programme eligibility, geographical characteristics, baseline information, and contextual similarities.
-
Applying covariate matching, propensity score matching, and weighting approaches to improve observed characteristics balance.
-
Using multivariable regression adjustment and appropriate model specifications to account for measured confounding factors.
-
Examining common support, covariate balance, missing data, model dependence, and residual differences that may undermine comparability.
Module 5: Difference-in-Differences and Longitudinal Comparisons
-
Understanding the difference-in-differences design and its application to programmes with intervention and comparison groups observed over time.
-
Assessing the parallel trends assumption through pre-intervention data, graphical inspection, contextual reasoning, and suitable diagnostic tests.
-
Specifying regression models with group and time effects while considering treatment timing, clustering, and changes in group composition.
-
Investigating anticipation effects, spillovers, time-varying confounding, heterogeneous treatment effects, and alternative explanations for observed changes.
Module 6: Interrupted Time Series and Synthetic Control Methods
-
Designing interrupted time-series assessments using repeated observations before and after programme introduction or policy implementation.
-
Estimating changes in outcome levels and trends while accounting for seasonality, autocorrelation, concurrent events, and measurement changes.
-
Understanding synthetic control methods for constructing weighted comparison units from unaffected locations or organizations with historical outcome data.
-
Evaluating donor pool suitability, pre-intervention fit, placebo comparisons, sensitivity checks, and limitations of time-based counterfactual designs.
Module 7: Regression Discontinuity and Other Quasi-Experimental Opportunities
-
Identifying programme eligibility thresholds, administrative cut-offs, and allocation rules that may support regression discontinuity designs.
-
Examining continuity assumptions, manipulation around thresholds, bandwidth selection, functional form, and local treatment effect interpretation.
-
Assessing instrumental variable opportunities where a defensible instrument and the required identification assumptions can be justified.
-
Matching evaluation design choices to institutional rules, programme allocation processes, ethical considerations, and the practical availability of data.
Module 8: Data Preparation, Diagnostics, and Sensitivity Analysis
-
Reviewing observational datasets for completeness, consistency, measurement quality, coding errors, missingness, and appropriate variable construction.
-
Conducting descriptive analysis, baseline comparisons, overlap checks, balance diagnostics, and graphical exploration before effect estimation.
-
Testing the robustness of findings through alternative specifications, placebo tests, falsification checks, and sensitivity to unobserved confounding.
-
Interpreting confidence intervals, standard errors, statistical significance, practical importance, and uncertainty without overstating precision.
Module 9: Mixed-Methods Triangulation and Causal Interpretation
-
Integrating quantitative estimates with interviews, programme records, implementation histories, and contextual evidence to assess competing explanations.
-
Using contribution analysis, process tracing, and programme theory to investigate whether estimated changes are consistent with expected causal pathways.
-
Examining implementation variation, participant experiences, external shocks, spillovers, and unintended effects when interpreting programme estimates.
-
Distinguishing causal identification from supporting explanatory evidence and documenting unresolved uncertainty and alternative interpretations.
Module 10: Reporting, Decision-Making, and Practical Application
-
Preparing transparent counterfactual evaluation reports that specify estimands, data sources, design choices, assumptions, analytical methods, and limitations.
-
Presenting estimated programme effects through accessible tables, graphs, effect-size summaries, and explanations of uncertainty for nontechnical audiences.
-
Translating evaluation findings into evidence-based recommendations for programme continuation, redesign, targeting, adaptation, and resource allocation.
-
Developing a practical nonexperimental assessment plan that includes a causal question, identification strategy, data requirements, diagnostics, and reporting framework.
Training Approach
This course will be delivered by our skilled trainers who have vast knowledge and experience as expert professionals in the fields. The course is taught in English and through a mix of theory, practical activities, group discussion and case studies. Course manuals and additional training materials will be provided to the participants upon completion of the training.
Tailor-Made Course
This course can also be tailor-made to meet organization requirement. For further inquiries, please contact us on: Email: training@upskilldevelopment.com Tel: +254 721 331 808
Training Venue
The training will be held at our Upskill Training Centre. We also offer training for a group (at a discount of 10% to 50%) at requested location all over the world. The Onsite course fee covers the course tuition, training materials, two break refreshments, buffet lunch, airport transfers, Upskill gift package, and guided tour.
Visa application, travel expenses, dinners, accommodation, insurance, and other personal expenses are catered by the participant
Certification
Participants will be issued with Upskill certificate upon completion of this course.
Airport Pickup and Accommodation
Airport pickup and accommodation is arranged upon request. For booking contact our Training Coordinator through Email: training@upskilldevelopment.com, +254 721 331 808
Terms of Payment:
Unless otherwise agreed between the two parties’ payment of the course fee should be done 3 working days before commencement of the training so as to enable us to prepare better.








