DANIDA Midterm evaluation Consultant

PROJECT SUMMARY 
Type of Evaluation    Mid-term evaluation
Name of the Project   DANIDA SPA Education and Child Protection Programme in Yemen 2022 – 2025
Project Start and End Dates   01 January 2022– 31 December 2025
Project Duration   4 years (48 months)
Project Locations:   Yemen – Sa’ada
Districts: Razih and Hydan.
Thematic Areas   Child protection
Sub-themes   Child protection and education
Donor   DANIDA
Overall Objective of the Project   Girls and boys of school age (between 6-17) are safe and protected from violence, natural and everyday hazards and conflict in and around schools and have access to safe and inclusive learning environments.

 
Project Overview 
The DANIDA SPA Education and Child Protection Programme in Yemen is being implemented in Sa’ada Governorate which is one of the areas most affected by the conflict. As per the latest Education Cluster data, 273.873 school-aged children (53% boys) are in need of humanitarian assistance.  Humanitarian actors have a limited presence in Sa’ada due to active fighting and access challenges – frontline. This has led to very few national and international organisations working in education and Child Protection (CP) programmes despite massive needs in the governorate. Based on the Education Cluster severity analysis, all districts (except Razih) in Sa`ada are considered priority 4 with a total number of 139,519 school-aged boys and girls in acute need.

The program outcome indicators:
1.   Local civil society, local authorities, community members and school personnel are engaged in processes aiming to develop, strengthen, and implement policies and systems for school safety and protection. 
2.   Teachers, students, parents, school administration and relevant duty bearers take responsibility for school facilities’ standards of safety, protection and management aimed at protecting children - in and around schools.
3.   Teachers, children and community members have increased capacity to develop safe and protective environments and promote positive relationships in and around schools 
4.   Children with child protection concerns have access to multi sector services and lifesaving information through coordinated case management services and referral mechanisms 
5.   Girls’ and boys’ voices are considered and echoed in decision-making process by children, humanitarian actors and other stakeholders.

This Mid-term evaluation, through comparison to the baseline findings, monitoring reports and the progress achieved in terms of the IPTT, will come up with an analysis of actions’ impacts on beneficiaries and sound recommendations on how to improve Save the Children programming while maximizing benefits to the supported communities.
SCOPE OF STUDY
This study will be conducted at the third year of the project (2024), to assess the project outcomes and ensure to which extent that the project met its impact as stipulated in the logframe. It will be built upon the baseline results, and benchmark indicators assessment and/or IPTT achievements, Activity completion reports and any other relevant reports.
The objectives of this mid-term evaluation are to:
WHY:
•   Assess the extent to which the project met its outcomes as stipulated in the project Logframe.
•   Assess the extent of which the project accessed girls, boys, women and men equally in the project activities and services.  
•   Assess the extent to which project implementation progress contributes and meet the Safe School Common Approach guidelines and standards.
•   Assess which approaches and activities are working well and where we should make changes.
•   Assess the satisfaction of girls, boys and their caregivers from the case management intervention against the case assessment and plans and their perception of how the project contributed to their needs; and
•   Highlight lessons learned (including any unintended outcomes) and recommendations to feedback into current and future SC programming.
•   Assess how and why the intervention may have made a difference to the targeted women, girls, boys and men (separately).
WHAT: 
•   The Mid-term evaluation will consider all above objectives against program implementation in the targeted schools in Sa’ada governorate.
•   The Mid-term evaluation is planned to cover the assessment of the implemented activities throughout the project pervious period (2022 and 2023). 
WHO & HOW: 
The primary audiences of this evaluation are the donor DANIDA, Save the Children Member Office and Country Office, Ministry of Education and other project stakeholders, who will use the assessment of program outcomes to inform their evidence-based programming and learning agendas, as well as the evaluation recommendations to inform relevant programmatic and operational processes as well as future program design.

Area of Evaluation   Overarching objectives/questions
Implementation/
Process      How well was the program/project implemented? (see key study questions regarding fidelity and process)
   Did program/project beneficiaries (women, girls, boys and men) have a positive experience with the program? (see key study questions regarding satisfaction and experience)
   Did the program/project activities meet the beneficiaries (women, girls, boys and men) preferences and needs?
   How and why did the intervention make a difference?
Outcome      Did the program/project achieve its intended outcomes? (see key study questions regarding impact, effectiveness and sustainability)
   Did the program support women, girls, boys and men to develop self-confidence and effective decision-making with regards to the program activities and outcomes? 
Impact      What positive and negative direct or indirect, intended or unintended effects can be attributed to the programme? [DAC definition]
   What positive and negative direct or indirect, intended or unintended effects on women, girls, boys, and men in the targeted community so far?

The selected consultant will be required to undertake external consultation with the HoM/CO MEAL specialist and the PDQ TAs to further refine the Study questions.
Key Study Questions:
Below questions are to facilitate and guide defining the methodology, as well as to be more focused in study findings.
Criteria   Key Study Questions   Process   Outcome   Impact
Accountability      How has the program/project approached accountability to girls and boys and the wider community?
   How do girls and boys and community members prefer to be reached, and what is the best modality of communication?
   How do girls, boys, women and men are participating in the project and how their feedback and suggestions are reflected in the project?    X   X   X
Child participation      How have the girls and boys, their needs, desires and suggested solutions, been consulted, accounted for and reflected in programme or project design and implementation?
   How were girls and boys supported to meaningfully participate across the programme/project cycle? What effective mechanisms were used? Were their feedback and complaints informed the implementation approaches and modalities? 
   How were feedabcks of boys and girls responded to?    X   X   X
Child rights programming      How has the program/project design and implementation considered a child rights approach? 
   Did girls and boys (equally) as well as female and mela community members develop knowledge about children rights?   X   X   
Safe programming      Has the program been designed, planned, implemented and monitored to ensure it is equally safe for girls, boys, women and men? 
   How has child safety been integrated into the program/project design and implementation of activities? What aspects of the program/project make girls and boys feel safe?
   How has the program/project assessed the risks for girls and boys (separately) and do these risks still exist to date? Have they been reduced, controlled and managed by the minimizing actions? Are there new risks?  What further measures do we need to implement to reduce, remove and control these new emerging risks?
   How did the program activities prevented, mitigated and responded to Gender-based Violence (GBV) against women, girls, boys and men?    X   X   
Equity and equality      Did/does the intervention have an impact on inequality, gender, or marginalization? 
   What mechanisms / factors contributed to ensure social equality and behavioural change on negative social norms in the community?
   How did women (as community members such as caregivers, or service providers such as teachers and social workers) participate equally with men in project activities and project decision-making? How did their participation inform the program design and implementation effectively?       X   X
Fidelity      Did implementation meet quality standards / best practice?
   What were the barriers and facilitators to implementation?   X   X   X
Impact*      Does the program/project contribute to reaching higher level objectives (preferably, overall objective)?  Why/ why not?
   What is the impact or effect of the programme or project in proportion to the overall situation of the target group or those effected (especially women and girls)? 
   What are the intended or unintended effects of the programme, either positive or negative, direct or indirect?         X
Inclusion      How did the program/project consider inclusion of vulnerable groups in the design and its implementation of activities? 
   Did the program address different needs of girls, boys, women and men from IDPs, host community, disability/without disability, and from minority groups equally?
   How were girls, boys, women and men with disabilities engaged and benefitted from the program services equally with those without disabilities?  
   How did girls, boys, women and men with disabilities and from minority groups participate effectively in the project and how their participation informed the program design and implementation?    X   X   X
Process      How well did staff/partnerships work together?
   How can implementation of the program be improved in terms of coordination with different stakeholders?
   Have MEAL activities been implemented as planned? Were they relevant and appropriate? How could they be improved?   X   X   
Reach and uptake      Did the program/project reach its intended target populations (women, girls, boys and men) equally? Why/ why not?
   How do girls and boys/families who accessed and completed the program differ from those who did not?
   What were the barriers and facilitators to women, girls, boys and men accessing/
completing the program?   X   X   X
Replicability and scale      Are the evaluation findings generalizable to other contexts?
   Will the project or programme work in a different context?       X   X
Satisfaction and experience      How satisfied were the program/project beneficiaries (women, men, girls and men- separately)?
   Did program beneficiaries (women, men, girls and boys- separately) feel the services they received were acceptable, appropriate, and suited to their needs?   X   X   
Gender sensitivity      How has the program/project considered gender sensitivity both in the design and its implementation of activities?
   Has the program/project incorporated different needs and accessibility of boys and girls, men and women? 
   Has the program/project outcomes or results been equally represented? 
   What are the gender gaps that the program/project addressed and what remaining aspects need to be considered further?
   How were the project activities tailored according to the different needs and the effective participation to women, girls, boys, and men separately?    X   X   X

The evaluation will be based on the selective OECD-DAC criteria and is also expected to consider Save the Children quality benchmarks for high-impact responses and programs.

EXPECTED DELIVERABLES
The study deliverables and tentative timeline (subject to the commencement date of the study) are outlined below. Head of MEAL and MEAL specialist will agree on final milestones and deadlines at the inception phase.
Deliverables and Tentative Timeline
Deliverable / Milestones   Timeline
The consultant is contracted and commences work   7.6.2024
The consultant will facilitate a workshop with the relevant stakeholders at the commencement of the project to develop the inception report.   11.6.2024
The consultant will submit an inception report* in line with the provided template, including:
   Study objectives, scope and key study questions
   description of the methodology, including design, data collection methods, sampling strategy, data sources, and study matrix against the key study questions
   data collection, analysis and reporting plan
   caveats and limitations of study 
   risks and mitigation plan
   ethical considerations including details on consent.
   stakeholder and children communication and engagement plan
   key deliverables, responsibilities, and timelines 
   resource requirements
   data collection tools (in line with the study matrix), calibration and it's validation plan.
   Considerations to ensure the participation of women, girls, boys, and men with and without disabilities.
   Once the report is finalised and accepted, the consultant must submit a request for any change in strategy or approach to the study manager or the steering committee.   15.6.2024
Ethics submission (if applicable):
Should approval from a Human Research Ethics Committee be required, an ethics submission should include:
   study protocols (participant recruitment, data security and storage, consent and confidentiality etc.)
   considerations for consulting with children and other vulnerable groups (if applicable)
   participant information statement and consent forms   18.6.2024
Final data collection tools (in the report language):
   Survey instrument
   Data collection mechanism   22.6.2024
An [Power Point Presentation] including a summary of formative findings from the study. The focus will be on:
   Summary of interim findings 
   Any emerging program issues or risks (if applicable)
   Any changes that have had to be made to the study design (if applicable)
   Key tasks for the next stage of the study and any proposed refinements or changes to methodology (if applicable)   25.6.2024
A Study Report* (Draft Version – template available if useful though external actors may want to use theirs) including the following elements: 
   Executive summary
   Background description of the Program and context relevant to the Study
   Scope and focus of the study.
   Overview of the study methodology and data collection methods, including a Study matrix
   Findings aligned to each of the key Study questions.
   The report analysis and report writing should demonstrate data about women, girls, boys and men separately as well as people with disabilities and other minority groups in the community.
   Specific caveats or methodological limitations of the evaluation 
   Conclusions outlining implications of the findings or learnings.
   Recommendations
   Annexes (Project logframe, study ToR, Inception Report, Study schedule, List of people involved)
A consolidated set of feedback from key stakeholders will be provided by Save The Children within 1 weeks of the submission of the draft report.   30.6.2024
Data and analyses including all encrypted raw data, databases and analysis outputs based on gender, age and disability   6.7.2024
Final Study Report* incorporating feedback from consultation on the Draft Study Report   16.7.2024
Knowledge translation materials:
   PowerPoint presentation of Study findings
   Evidence to Action Brief**   31.7.2024

QUALIFICATIONS AND EXPERIENCE:
The following qualifications and skills are expected of the lead consultancy:
Assigned staff must have:
o   An advanced university degree in Psychology, Education and systems management or related field.
o   At least 7 years’ experience in the area of Child Protection and Education.
o   Technical expertise in evaluation research, preferably integrated interventions mainly Child Protection and Education.
o   Technical expertise in evaluation cross-cutting areas in interventions such as gender sensitivity, social equity, disability inclusion and child participation. 
o   Previous experience in MEAL, including conducting Mid-term and end-of-project evaluations for large-scale projects.
o   Thorough understanding of data collection methods
o   Strong skills in quantitative and qualitative methods 
o   Strong interpersonal and communication skills;
o   Fluency in English is a must, fluency in Arabic (preferred) is an added advantage;
o   Experience/knowledge on gender-sensitive programming
o   Understanding of child safeguarding and child participation procedures
o   Experience of working in the middle east and/ or Yemen is a plus
o   The consultant should get the permission from SCHAMCHA to conduct the midterm evaluation.
Expected outputs
The expected outputs of this evaluation are as follows:
a)   Inception report (not exceeding 30 pages)
b)   methods and tools
c)   data sets
d)   Final report. (with an executive summary of max 2 pages) 
e)   Power point presentation of the findings (not exceeding 15 slides)
f)   Presentation of findings to key stakeholders
The Organisation
We employ approximately 25,000 people across the globe and work on the ground in over 100 countries to help children affected by crises, or those that need better healthcare, education and child protection. We also campaign and advocate at the highest levels to realise the right of children and to ensure their voices are heard.  

We are working towards three breakthroughs in how the world treats children by 2030:
•   No child dies from preventable causes before their 5th birthday 
•   All children learn from a quality basic education and that,
•   Violence against children is no longer tolerated

We know that great people make a great organization, and that our employees play a crucial role in helping us achieve our ambitions for children.  We value our people and offer a meaningful and rewarding career, along with a collaborative and inclusive environment where ambition, creativity, and integrity are highly valued. 
 

The Organization
We employ approximately 25,000 people across the globe and work on the ground in over 100 countries to help children affected by crises, or those that need better healthcare, education and child protection. We also campaign and advocate at the highest levels to realise the right of children and to ensure their voices are heard. We are working towards three breakthroughs in how the world treats children by 2030:
· No child dies from preventable causes before their 5th birthday
· All children learn from a quality basic education and that,
· Violence against children is no longer tolerated
We know that great people make a great organization, and that our employees play a crucial role in helping us achieve our ambitions for children.We value our people and offer a meaningful and rewarding career, along with a collaborative and inclusive workplace where ambition