Course Introduction
Propensity Score Methods and Covariate Balancing for Evaluation is a specialised professional training course designed to strengthen participants’ ability to estimate programme and policy effects using observational data. In many public sector and development settings, programmes are targeted towards particular populations, and participation is influenced by income, location, eligibility criteria, institutional capacity, previous performance or individual motivation. These differences can make direct comparisons between participants and non-participants misleading. The course equips participants with the analytical knowledge and practical techniques needed to address observed confounding, construct more comparable evaluation groups and improve the credibility of impact estimates when randomised assignment is unavailable or infeasible.
The course introduces the theoretical foundations of propensity score analysis, including the probability of receiving an intervention conditional on observed baseline characteristics. Participants examine how propensity scores can support matching, inverse probability weighting, stratification, subclassification and covariate adjustment. Particular attention is given to the distinction between estimating the probability of treatment and demonstrating that the resulting comparison groups are sufficiently balanced. Participants learn that a well-performing treatment prediction model does not necessarily produce a credible impact evaluation, and that method selection must be informed by the research question, target population, assumptions and characteristics of the available data.
A central component of the training is covariate selection and balance assessment. Participants learn to identify relevant pre-intervention characteristics using programme theory, institutional knowledge, causal diagrams and prior empirical evidence. They explore how to distinguish confounders from instrumental variables, colliders and variables affected by the intervention, reducing the risk of introducing bias through inappropriate adjustment. The course develops practical skills in measuring standardised mean differences, comparing variances and distributions, inspecting propensity score overlap and interpreting balance plots. Participants also learn how to refine analytical specifications when important imbalances remain and how to document these decisions transparently.
The programme explores several approaches to constructing comparison groups and estimating intervention effects. Participants examine nearest-neighbour and caliper matching, matching with and without replacement, full and optimal matching, inverse probability of treatment weighting, stabilised weights, overlap weighting and related balancing methods. The curriculum also introduces propensity score stratification and doubly robust estimation, explaining the circumstances in which these approaches may be useful. Rather than treating any method as universally superior, participants learn to evaluate trade-offs involving bias, variance, sample retention, extreme weights, common support and the population to which the findings can reasonably be generalised.
Advanced sessions focus on the practical challenges that often determine whether propensity score evaluations produce defensible results. These include limited overlap, positivity violations, missing baseline information, treatment selection driven by unobserved factors, high-dimensional covariates, nonlinear relationships and heterogeneous intervention effects. Participants learn to conduct sensitivity analyses, examine alternative specifications, quantify uncertainty appropriately and distinguish statistical precision from causal credibility. The training also addresses the limitations of propensity score methods: balancing observed covariates cannot, by itself, eliminate bias from unmeasured confounding or compensate for poor outcome measurement, inappropriate study design or a lack of comparable units.
By the end of the course, participants will be equipped to develop, implement and critically appraise propensity score-based evaluation strategies for public policies, development programmes, social interventions and institutional reforms. They will be able to justify their choice of estimand, select suitable balancing methods, assess whether observed covariates have been adequately balanced and communicate treatment effect estimates with appropriate qualifications. Practical exercises and an integrated evaluation project will help participants translate statistical output into clear, decision-relevant evidence. The programme emphasises reproducibility, analytical transparency and responsible interpretation so that findings can inform programme improvement, resource allocation and policy decisions without overstating what observational data can establish.
Duration
10 days
Who Should Attend
-
Monitoring and evaluation specialists responsible for measuring programme effectiveness and assessing development intervention outcomes.
-
Government policy analysts who evaluate public policies, subsidies, social assistance schemes and administrative reforms using observational data.
-
Economists and impact evaluation researchers undertaking quasi-experimental studies and causal inference analyses.
-
Statisticians and quantitative analysts seeking advanced skills in propensity score estimation, weighting and covariate balancing.
-
Development practitioners evaluating projects funded by governments, international development agencies, foundations and non-governmental organisations.
-
Public health researchers assessing treatment, service delivery, prevention and health system interventions using non-randomised data.
-
Education researchers evaluating school improvement initiatives, student support programmes, teacher development and education financing reforms.
-
Social protection specialists analysing the outcomes of cash transfers, livelihood programmes and targeted assistance initiatives.
-
Agricultural and rural development professionals evaluating extension services, input subsidies, farmer training and market access interventions.
-
Financial inclusion and enterprise development practitioners assessing credit programmes, business support schemes and entrepreneurship initiatives.
-
Research consultants and evaluation firms designing or reviewing quantitative impact assessment studies.
-
University lecturers, doctoral candidates and postgraduate researchers conducting applied econometric and programme evaluation research.
-
Programme managers and technical advisers responsible for commissioning, supervising or interpreting impact evaluation assignments.
-
Data managers and administrative information specialists preparing baseline covariates, treatment indicators and outcome datasets.
-
Donor representatives, technical reviewers and senior decision-makers who need to assess the credibility and limitations of propensity score-based findings.
Course Objectives
-
Explain the theoretical foundations of propensity score methods and describe how they can improve comparability between intervention and comparison groups in observational evaluation studies.
-
Formulate clear causal research questions and select appropriate estimands, including the average treatment effect and average treatment effect on the treated, according to the policy decision being supported.
-
Identify relevant pre-intervention covariates using causal reasoning, programme knowledge and empirical evidence while avoiding inappropriate adjustment for post-treatment variables, colliders or unsuitable instruments.
-
Estimate propensity scores using appropriate statistical models and assess whether the specification captures important relationships between baseline characteristics and intervention assignment.
-
Apply matching, stratification, inverse probability weighting and covariate adjustment methods, explaining the assumptions, benefits and limitations associated with each analytical approach.
-
Assess common support, overlap and positivity, identifying situations in which treated and comparison units are too dissimilar to support reliable estimation for the intended target population.
-
Measure covariate balance using standardised mean differences, variance ratios, graphical diagnostics and distributional comparisons before and after applying the selected balancing method.
-
Diagnose extreme propensity scores, influential weights, limited effective sample sizes and residual covariate imbalance, selecting defensible remedies without obscuring important weaknesses in the underlying data.
-
Estimate treatment effects using suitable outcome models, weighting procedures and doubly robust approaches, ensuring that uncertainty estimation reflects the design and dependence structure of the data.
-
Conduct sensitivity analyses and robustness checks to examine the effects of alternative model specifications, matching choices, overlap restrictions and plausible unmeasured confounding.
-
Implement reproducible analytical workflows in appropriate statistical software, documenting data preparation, model specifications, balance diagnostics, sample exclusions and treatment effect estimation.
-
Prepare transparent evaluation reports that explain the analytical strategy, balance achieved, estimated effects, uncertainty, limitations and implications for programme management and policy decisions.
Comprehensive Course Outline
Module 1: Foundations of Propensity Score Methods
-
Define the propensity score as the conditional probability of intervention assignment given observed baseline characteristics, and explain its role in constructing comparable groups for observational evaluation.
-
Examine how selection bias and confounding arise when programme participants differ systematically from non-participants in characteristics associated with both participation and outcomes.
-
Compare propensity score methods with conventional regression adjustment, randomised controlled trials and other quasi-experimental approaches, identifying their distinct assumptions and intended applications.
-
Review the practical strengths and limitations of propensity score methods in public policy, health, education, agriculture, social protection and development programme evaluation.
Module 2: Causal Questions, Estimands and Evaluation Design
-
Translate programme objectives into clearly specified causal questions identifying the intervention, comparison condition, target population, outcome measures and relevant follow-up period.
-
Distinguish the average treatment effect, average treatment effect on the treated and other population-specific estimands, explaining how each corresponds to a different decision-making objective.
-
Define treatment assignment, eligibility, baseline covariates and outcome measurement consistently, ensuring that the analytical dataset reflects the intended evaluation design.
-
Develop an evaluation protocol that documents the causal assumptions, target population, planned balancing method, outcome analysis and criteria for assessing the credibility of findings.
Module 3: Covariate Selection and Causal Reasoning
-
Identify potential confounders using programme theory, institutional knowledge, prior evidence and causal diagrams that clarify relationships between treatment assignment and the outcome.
-
Distinguish pre-treatment confounders from variables affected by the intervention, mediators, colliders and instrumental variables that may introduce bias if included indiscriminately.
-
Evaluate nonlinear relationships, interactions and contextual factors that influence participation and outcomes, selecting covariates based on causal relevance rather than predictive significance alone.
-
Prepare and document a covariate selection strategy that considers measurement timing, data availability, substantive importance and the consequences of omitting important confounders.
Module 4: Data Preparation and Exploratory Assessment
-
Structure individual-level, household-level, firm-level or institutional datasets with consistent treatment indicators, baseline characteristics, outcome variables and unique observation identifiers.
-
Assess missing data, inconsistent coding, duplicate observations, extreme values and measurement differences that may compromise the estimation of propensity scores or treatment effects.
-
Compare baseline characteristics across intervention and comparison groups using descriptive statistics, distributions and graphical summaries before any adjustment is applied.
-
Develop a data preparation record covering inclusion criteria, variable definitions, recoding decisions, missingness patterns and reproducible procedures for generating the analytical sample.
Module 5: Propensity Score Estimation and Model Specification
-
Estimate propensity scores using logistic regression, probit regression or other appropriate models that represent the conditional probability of receiving the intervention.
-
Specify relevant covariates, nonlinear terms and theoretically justified interactions, recognising that the best assignment model is not necessarily the one with the highest predictive accuracy.
-
Examine predicted score distributions, extreme predicted probabilities and model stability to identify separation, overfitting and possible weaknesses in the treatment assignment specification.
-
Refine the propensity score model using observed balance and substantive knowledge, documenting the rationale for changes and avoiding specifications selected solely to produce a preferred treatment effect.
Module 6: Covariate Balance Diagnostics and Assessment
-
Calculate standardised mean differences for baseline covariates before and after adjustment, interpreting the direction and magnitude of residual imbalance rather than relying on significance tests alone.
-
Assess variance ratios, distributional differences and higher-order moments to determine whether balancing procedures improve more than just the average values of selected covariates.
-
Produce balance tables, Love plots, propensity score density plots and other graphical diagnostics that communicate the quality of comparability between intervention and comparison groups.
-
Establish transparent balance assessment criteria, identify important residual imbalances and document whether further model refinement or a different analytical method is warranted.
Module 7: Propensity Score Matching Techniques
-
Implement nearest-neighbour matching and evaluate how matching order, distance metrics and matching ratios influence sample composition, balance and the precision of treatment effect estimates.
-
Apply caliper restrictions and alternative distance measures to reduce poor-quality matches, assessing the balance improvement against the loss of potentially useful observations.
-
Compare matching with and without replacement, optimal matching and full matching, explaining their implications for control reuse, sample retention and representation of the target population.
-
Evaluate matched sample quality through post-matching diagnostics, record unmatched observations and exclusions, and estimate treatment effects using methods appropriate to the matching design.
Module 8: Stratification, Subclassification and Matching Alternatives
-
Divide observations into propensity score strata and examine whether treated and comparison units are sufficiently similar within each stratum to support meaningful outcome comparisons.
-
Estimate stratified treatment effects and combine stratum-specific estimates using weights aligned with the selected target population and causal estimand.
-
Compare subclassification, coarsened matching and other matching approaches according to their balance performance, transparency, computational demands and applicability to the evaluation question.
-
Investigate whether estimated effects vary across strata, recognising that apparent differences may reflect genuine treatment effect heterogeneity, residual imbalance or limited data within particular groups.
Module 9: Inverse Probability Weighting and Weight Diagnostics
-
Derive inverse probability of treatment weights for treated and comparison units and explain how these weights construct a pseudo-population with improved balance on observed baseline characteristics.
-
Distinguish weighting schemes designed for the average treatment effect from those targeting the treated population or other clearly defined groups of interest.
-
Diagnose extreme weights, influential observations and reduced effective sample size, evaluating the implications for variance, precision, overlap and the stability of estimated intervention effects.
-
Examine stabilised weights, justified weight truncation and alternative weighting strategies, documenting how each decision affects balance, sample representation and interpretation of the target estimand.
Module 10: Covariate Balancing Weighting and Overlap Methods
-
Introduce overlap weighting and related approaches that emphasise observations with meaningful representation in both intervention and comparison groups when estimating effects for the population in overlap.
-
Explain the purpose of covariate balancing propensity score approaches, which incorporate balance-related criteria into score estimation rather than relying exclusively on treatment prediction.
-
Compare conventional inverse probability weighting, overlap weighting and other balancing methods using standardised differences, effective sample sizes and the characteristics of the resulting weighted population.
-
Select an appropriate balancing approach according to the causal question, overlap limitations and desired target population, explaining why a method that improves balance may still change the population to which findings apply.
Module 11: Doubly Robust Estimation and Outcome Modelling
-
Explain the logic of doubly robust estimators that combine a treatment assignment model with an outcome regression model under their relevant identification and regularity assumptions.
-
Implement augmented inverse probability weighting or other suitable doubly robust procedures, distinguishing the roles of propensity score estimation, outcome modelling and treatment effect calculation.
-
Examine how residual imbalance, model misspecification, limited overlap and influential observations affect the reliability of estimates even when doubly robust methods are applied.
-
Compare results from weighting, matching and outcome-adjusted analyses, explaining differences in estimands, precision and assumptions rather than selecting the estimate that appears most favourable.
Module 12: Overlap, Positivity and Common Support
-
Explain positivity and common support as requirements for credible comparisons, recognising that meaningful treatment effects cannot reliably be estimated for groups with no suitable counterfactual observations.
-
Inspect propensity score distributions and covariate profiles to identify areas where treated and comparison observations have limited overlap or substantially different baseline characteristics.
-
Evaluate trimming, restriction to a common support region, alternative weights and overlap-focused estimands, documenting the population exclusions and their substantive consequences.
-
Assess whether the revised target population remains relevant to policy decisions and avoid generalising estimates to groups that are poorly represented in the adjusted analysis.
Module 13: Missing Data, High-Dimensional Covariates and Advanced Methods
-
Assess patterns and potential causes of missing baseline covariates, distinguishing missingness that can be addressed through defensible assumptions from situations requiring substantial caution.
-
Examine appropriate missing data strategies, including multiple imputation where justified, and explain how imputation procedures should be integrated with propensity score estimation and balance assessment.
-
Explore high-dimensional covariate settings and flexible estimation techniques, considering regularisation, machine learning and the need to avoid overfitting or compromising the intended causal estimand.
-
Evaluate generalised propensity score approaches for multi-category treatments or continuous treatment intensity, identifying additional assumptions and analytical requirements beyond the binary-treatment setting.
Module 14: Sensitivity Analysis, Unmeasured Confounding and Robustness
-
Explain why propensity score methods address confounding only through measured and appropriately modelled baseline covariates, leaving possible bias from unobserved determinants of treatment and outcomes.
-
Conduct sensitivity analyses using alternative score specifications, matching algorithms, calipers, weighting schemes and overlap restrictions to investigate whether findings depend on specific analytical choices.
-
Examine methods for assessing the possible influence of unmeasured confounding, including quantitative sensitivity analyses and other approaches suitable for the research design and available evidence.
-
Develop a structured robustness assessment that reports changes in estimates, uncertainty, balance and sample composition without treating agreement across methods as proof that causal assumptions hold.
Module 15: Treatment Effect Estimation, Inference and Reporting
-
Estimate treatment effects using outcome differences, regression-based procedures and appropriate weighted analyses, aligning the estimator with the prespecified causal estimand and target population.
-
Select suitable standard errors, confidence intervals and resampling approaches while accounting for the evaluation design, clustering, matched observations and estimated weights where relevant.
-
Interpret effect sizes, uncertainty intervals, practical significance and possible treatment effect heterogeneity, distinguishing statistical evidence from claims of programme effectiveness.
-
Prepare transparent evaluation reports documenting the treatment assignment model, covariates, balance diagnostics, overlap, sample exclusions, effect estimates, sensitivity analyses and limitations.
Module 16: Statistical Software Practicum and Integrated Evaluation Project
-
Prepare an observational evaluation dataset and implement a complete propensity score workflow using appropriate software such as R, Stata or another supported statistical analysis environment.
-
Estimate propensity scores, apply selected matching or weighting procedures, produce balance diagnostics and refine the analysis according to predefined evaluation criteria.
-
Calculate treatment effects and uncertainty measures, conduct sensitivity analyses and compare alternative specifications while preserving the distinction between primary and supplementary findings.
-
Present an integrated evaluation report and technical briefing that defend the analytical choices, explain remaining threats to validity and translate the evidence into practical programme recommendations.
Training Approach
This course will be delivered by our skilled trainers who have vast knowledge and experience as expert professionals in the fields. The course is taught in English and through a mix of theory, practical activities, group discussion and case studies. Course manuals and additional training materials will be provided to the participants upon completion of the training.
Tailor-Made Course
This course can also be tailor-made to meet organization requirement. For further inquiries, please contact us on: Email: training@upskilldevelopment.com Tel: +254 721 331 808
Training Venue
The training will be held at our Upskill Training Centre. We also offer training for a group (at a discount of 10% to 50%) at requested location all over the world. The Onsite course fee covers the course tuition, training materials, two break refreshments, buffet lunch, airport transfers, Upskill gift package, and guided tour.
Visa application, travel expenses, dinners, accommodation, insurance, and other personal expenses are catered by the participant
Certification
Participants will be issued with Upskill certificate upon completion of this course.
Airport Pickup and Accommodation
Airport pickup and accommodation is arranged upon request. For booking contact our Training Coordinator through Email: training@upskilldevelopment.com, +254 721 331 808
Terms of Payment:
Unless otherwise agreed between the two parties’ payment of the course fee should be done 3 working days before commencement of the training so as to enable us to prepare better.








