Course Introduction
Meta-evaluation is the systematic assessment of the quality, credibility, usefulness, and methodological rigour of evaluation studies. Organizations increasingly commission multiple evaluations to inform strategic planning, programme improvement, public accountability, and investment decisions. However, the existence of an evaluation report does not automatically guarantee that its methods are appropriate, its findings are credible, or its recommendations are actionable. This five-day training course equips participants with the knowledge and practical skills needed to assess evaluation quality systematically and determine whether evaluation evidence is sufficiently robust to support important organizational and policy decisions.
The course introduces the conceptual foundations, purposes, and applications of meta-evaluation across development programmes, public sector initiatives, humanitarian interventions, health systems, education programmes, and institutional performance assessments. Participants will distinguish meta-evaluation from systematic reviews, evidence synthesis, audit, peer review, and routine quality assurance. They will examine how evaluation quality can be assessed against established professional standards, evaluation criteria, ethical principles, and organizational requirements while recognizing that appropriate quality benchmarks depend on the evaluation purpose, context, resources, and intended users.
A central component of the programme focuses on reviewing evaluation design and methodological quality. Participants will learn to assess whether evaluation questions are clearly formulated, whether the chosen methodology is suitable, and whether sampling, data collection, analytical procedures, and interpretation are sufficiently transparent and rigorous. The course addresses quantitative, qualitative, and mixed-methods evaluations, including experimental, quasi-experimental, theory-based, participatory, and contribution-focused approaches. Participants will practise identifying methodological weaknesses, unsupported causal claims, inadequate triangulation, inconsistent reporting, and gaps between research questions, evidence, findings, conclusions, and recommendations.
The training provides a structured approach to developing meta-evaluation frameworks, review protocols, scoring rubrics, evidence extraction templates, and quality assessment matrices. Participants will explore how to define review questions, establish assessment criteria, develop rating scales, document supporting evidence, and distinguish critical weaknesses from minor reporting deficiencies. They will also examine inter-reviewer consistency, calibration procedures, reviewer bias, and transparent decision rules. These skills are particularly important when reviewing multiple evaluation reports produced by different consultants, implementing partners, research organizations, or departments using varied methods and reporting formats.
Ethical practice, stakeholder engagement, and the responsible communication of review findings are also emphasized throughout the course. Participants will consider confidentiality, conflicts of interest, independence, cultural responsiveness, inclusion, and the fair treatment of evaluators whose work is being reviewed. They will learn to communicate strengths and limitations constructively, support quality ratings with documented evidence, and avoid making judgements that extend beyond the available information. The course also addresses how meta-evaluation findings can reveal recurring weaknesses across an evaluation portfolio and help organizations strengthen commissioning processes, evaluation capacity, and institutional learning.
By the end of the programme, participants will be able to plan and conduct a structured quality review, assess the credibility and usefulness of evaluation evidence, synthesize findings across multiple studies, and formulate practical recommendations for improving evaluation systems. Applied exercises will help participants develop a defensible meta-evaluation protocol, assess sample evaluation reports, prepare quality review matrices, and communicate conclusions to decision-makers. The resulting competencies support stronger evaluation governance, better evidence-informed decisions, and more consistent standards of quality across organizational and programme evaluation portfolios.
Duration
5 days
Who Should Attend
-
Monitoring, evaluation, accountability, and learning professionals responsible for evaluation quality assurance.
-
Evaluation specialists who review internal, external, and commissioned evaluation reports.
-
Managers of evaluation units and institutional research departments.
-
Government planning and policy officers assessing the quality of programme evaluations.
-
Development agency staff responsible for reviewing donor-funded evaluation deliverables.
-
NGO and civil society professionals overseeing evaluation quality and organizational learning.
-
Research managers coordinating studies, evaluations, and evidence synthesis activities.
-
Independent evaluation consultants seeking structured approaches to peer review and quality assessment.
-
Programme directors and project managers who use evaluation findings for strategic decisions.
-
Quality assurance and organizational performance specialists.
-
Academic researchers and postgraduate students undertaking evaluation methodology research.
-
Humanitarian evaluation practitioners reviewing emergency response and recovery assessments.
-
Public sector auditors and oversight professionals working with evaluation evidence.
-
Grant and portfolio managers assessing the credibility of partner evaluation reports.
-
Institutional learning and knowledge management professionals supporting evidence-informed practice.
Course Objectives
-
Explain the purpose, principles, and applications of meta-evaluation and distinguish it from systematic reviews, audits, peer reviews, and evidence synthesis.
-
Develop structured meta-evaluation protocols with clearly defined review questions, quality criteria, evidence requirements, scoring procedures, and reporting arrangements.
-
Assess the alignment between evaluation questions, programme theories, evaluation designs, data collection strategies, analytical methods, and conclusions.
-
Evaluate quantitative, qualitative, and mixed-methods studies using criteria appropriate to their methodological approaches and intended purposes.
-
Apply established evaluation standards, ethical principles, and professional quality benchmarks when reviewing individual evaluation reports and portfolios.
-
Identify methodological weaknesses, unsupported conclusions, reporting gaps, bias risks, and limitations affecting the credibility or usefulness of evaluation findings.
-
Design quality assessment matrices, scoring rubrics, evidence extraction tools, and reviewer guidance that promote transparency and consistent judgements.
-
Assess reviewer agreement, manage conflicts of interest, and apply calibration procedures to improve the reliability and fairness of quality ratings.
-
Synthesize findings across evaluation studies to identify recurring strengths, weaknesses, evidence gaps, and opportunities for institutional improvement.
-
Produce clear meta-evaluation reports with evidence-based ratings, practical recommendations, and prioritized actions for strengthening evaluation quality.
Comprehensive Course Outline
Module 1: Foundations and Principles of Meta-Evaluation
-
Defining meta-evaluation, its purposes, core concepts, and applications in evaluation governance and institutional learning.
-
Distinguishing meta-evaluation from systematic reviews, research synthesis, audits, peer review, and routine quality assurance.
-
Examining professional evaluation standards, ethical principles, and quality expectations across different organizational settings.
-
Identifying the intended users, scope, decision needs, and practical constraints of a meta-evaluation assignment.
Module 2: Evaluation Standards, Criteria, and Quality Frameworks
-
Reviewing established evaluation standards and professional guidance to identify relevant criteria for assessing study quality.
-
Applying criteria related to relevance, coherence, effectiveness, efficiency, impact, sustainability, and other context-appropriate evaluation dimensions.
-
Assessing methodological rigour, credibility, transparency, usefulness, ethics, inclusion, and stakeholder participation in evaluation studies.
-
Adapting quality frameworks to different evaluation purposes, programme contexts, methodological approaches, and reporting requirements.
Module 3: Developing Meta-Evaluation Protocols and Review Questions
-
Formulating clear meta-evaluation questions that address methodological quality, reporting completeness, credibility, and practical usefulness.
-
Defining inclusion criteria, study selection procedures, document requirements, review scope, and assessment boundaries.
-
Preparing review protocols, evidence extraction forms, quality checklists, scoring rubrics, and reviewer instructions.
-
Establishing transparent rating categories, decision rules, documentation procedures, and arrangements for resolving review disagreements.
Module 4: Reviewing Evaluation Design and Methodological Rigour
-
Assessing the alignment between evaluation questions, programme logic, theory of change, research design, and intended conclusions.
-
Reviewing sampling approaches, participant selection, sample adequacy, representativeness, and potential sources of selection bias.
-
Evaluating the suitability of quantitative, qualitative, experimental, quasi-experimental, and mixed-methods approaches.
-
Examining validity, reliability, triangulation, analytical transparency, causal reasoning, and the treatment of uncertainty.
Module 5: Assessing Data Quality, Analysis, and Evidence Credibility
-
Reviewing data collection instruments, fieldwork procedures, data management practices, and documentation of data limitations.
-
Assessing the appropriateness of statistical analysis, qualitative coding, interpretation, and integration of different evidence sources.
-
Identifying unsupported claims, selective reporting, inconsistencies, analytical errors, and conclusions that exceed available evidence.
-
Examining missing data, measurement error, researcher influence, contextual limitations, and potential threats to credibility.
Module 6: Evaluating Ethics, Inclusion, and Stakeholder Engagement
-
Assessing informed consent, confidentiality, privacy protection, safeguarding, and responsible handling of evaluation information.
-
Reviewing the inclusion of gender, disability, marginalized populations, and other relevant equity considerations in evaluation design.
-
Examining stakeholder participation, community perspectives, cultural responsiveness, and the appropriate representation of affected groups.
-
Identifying conflicts of interest, threats to independence, ethical reporting concerns, and safeguards for reviewer impartiality.
Module 7: Quality Scoring, Reviewer Calibration, and Consistency
-
Developing practical rating scales that distinguish critical methodological weaknesses from minor documentation or presentation issues.
-
Applying quality assessment matrices and evidence-based scoring rules to evaluation reports and supporting documentation.
-
Conducting reviewer calibration exercises and examining inter-reviewer agreement to improve consistency across assessments.
-
Managing ambiguous evidence, divergent reviewer judgements, missing information, and disagreements about quality ratings.
Module 8: Portfolio-Level Meta-Evaluation and Evidence Synthesis
-
Organizing and comparing multiple evaluation studies using common criteria while preserving relevant methodological distinctions.
-
Identifying recurring quality weaknesses, methodological trends, evidence gaps, and strengths across an evaluation portfolio.
-
Synthesizing findings from diverse programmes, sectors, geographical contexts, and evaluation designs without inappropriate aggregation.
-
Translating portfolio-level review findings into lessons for commissioning, evaluation capacity development, and institutional learning.
Module 9: Reporting Findings and Developing Quality Improvement Recommendations
-
Structuring meta-evaluation reports with clear methods, evidence sources, assessment criteria, ratings, findings, and limitations.
-
Writing defensible judgements that connect each quality rating to documented evidence and transparent decision rules.
-
Prioritizing recommendations according to significance, feasibility, urgency, institutional responsibility, and expected improvement.
-
Presenting findings through executive summaries, quality scorecards, review matrices, and decision-oriented reporting formats.
Module 10: Applied Meta-Evaluation Project and Institutional Quality Assurance
-
Designing a complete meta-evaluation exercise using a defined review question, protocol, criteria, and evaluation report sample.
-
Conducting a practical quality review and documenting methodological strengths, weaknesses, evidence gaps, and justified ratings.
-
Developing an institutional action plan covering evaluation commissioning, report review, reviewer development, and quality monitoring.
-
Presenting meta-evaluation results and defending recommendations through a structured review meeting and peer feedback session.
Training Approach
This course will be delivered by our skilled trainers who have vast knowledge and experience as expert professionals in the fields. The course is taught in English and through a mix of theory, practical activities, group discussion and case studies. Course manuals and additional training materials will be provided to the participants upon completion of the training.
Tailor-Made Course
This course can also be tailor-made to meet organization requirement. For further inquiries, please contact us on: Email: training@upskilldevelopment.com Tel: +254 721 331 808
Training Venue
The training will be held at our Upskill Training Centre. We also offer training for a group (at a discount of 10% to 50%) at requested location all over the world. The Onsite course fee covers the course tuition, training materials, two break refreshments, buffet lunch, airport transfers, Upskill gift package, and guided tour.
Visa application, travel expenses, dinners, accommodation, insurance, and other personal expenses are catered by the participant
Certification
Participants will be issued with Upskill certificate upon completion of this course.
Airport Pickup and Accommodation
Airport pickup and accommodation is arranged upon request. For booking contact our Training Coordinator through Email: training@upskilldevelopment.com, +254 721 331 808
Terms of Payment:
Unless otherwise agreed between the two parties’ payment of the course fee should be done 3 working days before commencement of the training so as to enable us to prepare better.








