Terms of Reference

Terms of Reference (TOR)

Multi-Country Project Evaluation

Beyond the Finish Line Programme – SNV Bhutan, Nepal and Lao PDR, 2018-2022

Background

SNV in collaboration with the Institute of Sustainable Futures, University of Technology Sydney (ISF-UTS) and CBM Australia, the governments of Nepal, Bhutan and Lao PDR and local partners are implementing a multi-country project within a 5-year agreement with a combined investment of AUD $13,309.362. The project is supported by the Australian Government’s Water for Women (WfW) Fund  which aims to improve the health, gender equality and well-being of Asian and Pacific communities through inclusive, sustainable water, sanitation and hygiene (WASH) programs.  Through this, it intentionally seeks to “raise the bar in terms of gender and socially inclusive analysis, design and program delivery in WASH”, and in doing so to lead practice globally in terms of gender and social inclusion practice . 

SNV Netherlands Development Organisation

SNV is an international non-profit organization working in over 30 countries in Africa, Asia and Latin America, including in Bhutan, Nepal and Lao PDR. Our goal is poverty alleviation which we aim to achieve by providing sustainable access to basic services and generating income and employment opportunities in our focus sectors of Food, Energy and Water. We have developed trusted relationships with governments and local partners and our approach is underpinned by a focus on inclusive development, building local ownership and creating systemic change. Within WASH programmes our approach is based on the recognition that access to water and sanitation are human rights. SNV builds the capacity of government duty bearers and stakeholders in civil society and private sector to strengthen sustainable service systems for all.

1https://www.waterforwomenfund.org/en/

2Within this Fund, gender and social inclusion refer to the inclusion of any social variables, which may result in marginalisation from decision-making around, or access to the use of WASH facilities within the chosen context.  As this is the Water for Women Fund, this particularly refers to gender, but may also include age, ability or impairment, ethnicity, religion, indigeneity, sexual orientation, gender identity and expression (SOGIE) and so on.

Beyond the Finish Line - Inclusive and Sustainable Rural Water Supply Services in Nepal

https://snv.org/project/inclusive-and-sustainable-rural-water-supply-services-nepal

The project aims to use the opportunity of decentralization in Nepal to develop inclusive, sustainable and resilient rural water supply services and hygiene behaviour change communication for households, schools and health facilities (40,000 people, 30 institutions) in two districts as a role model for inclusive WASH services in two provinces of 6.5 million people.

Although Nepal has a high coverage of water supply (87%), only 25% of those systems are well functioning as per national monitoring indicators. The challenge is therefore, beyond the finish line, in sustaining hygiene behaviours as well as the quantity, quality, accessibility and reliability (QQAR) of water supply services. The burden of poor hygiene and water lies disproportionately with women/girls, poor, low-caste and ethnic minority groups.

Building on its experience in strengthening gravity-fed rural water supply services, SNV will engage with the new local bodies formulated in Nepal to address gender and social inclusion within WASH governance, water supply service systems, and hygiene promotion in both gravity-fed and groundwater areas.

Introduction 

SNV will undertake a final evaluation in Nepal as part of the multi-country project – Beyond the Finish Line, 2018-2022 as a summative assessment of project achievements and outcomes. These terms of reference present the objective, the methodology, the deliverables and responsibilities of this process.   As a final evaluation it will address both project impact and effectiveness to understand the difference the multi-country project has made and how well it has delivered its outcomes. It is also important that the evaluation consider the sustainability of outcomes along with equity, relevance, and efficiency considerations.

Whilst the evaluation is to be conducted externally to provide independence and transparency, it is intended to engage project teams and partners, through for example sense-making processes and facilitated reflection workshops.   To enable this, SNV are seeking applications for national consultants in Nepal to undertake data collection under the guidance of a remote Lead Evaluator and Gender Equality, Disability and Social Inclusion (GEDSI) specialist.

Objectives

The objectives of the evaluation are to:

• Evaluate the impact, effectiveness, equity and sustainability of the three projects’ outcomes
• Support learning and improvement,
• Document and share knowledge from implementation . 

Methodology

The evaluation will use suitable qualitative research methods, including review of project documentation and literature. In consultation with SNV and the Lead Evaluator, the national consultants will develop an evaluation plan that responds to the evaluation questions, data collection methods (FGDs, KIIs) and includes a detailed planning and a division of roles and tasks between the Lead evaluators, SNV and the national consultants which will be contracted by SNV.

The lead evaluator supported by a dedicated gender and social inclusion specialist will design the approach to respond to the evaluation questions and then remotely support the local contracted in-country teams (Bhutan, Nepal)  who will undertake the agreed further data collection and analysis. The lead evaluator will then provide the final review and assessment of findings to meet the evaluation objectives with input from the GEDSI specialist. 

3Water for Women Fund: Mid-Term Review Guidance, August 2020

4Two consultancy teams will be contracted locally in Nepal and Bhutan by SNV to collect any additional data, whilst in Lao this will be undertaken by the Lead Evaluator/team with inputs from the GESI Specialist supported by local translators asrequired

The evaluation is expected to draw on existing performance monitoring data which has been collected in line with SNVs Performance Monitoring Guidelines  by SNV teams in October 22 and is not intended to duplicate survey processes. This includes existing household and institution level surveys (baseline, mid-term and endline), annual capacity assessments with government line agencies and SMEs, FGDs with key groups (e.g. People with disabilities, women, households living in poverty) as well as project deliverables and knowledge and learning products.

Evaluation questions (from the M&E Plans)

Impact: How has the project contributed to increased equitable access to and use of WASH services? Has progress been equitable?  What service levels were achieved?   Were outcomes as expected? Why? Why not? What project strategies were most effective?

Impact: How has the project furthered gender equity and social inclusion/or transformation?

Evidence of uptake of practice, innovation? Successful and unsuccessful outcomes (positive and negative)?

Sustainability: To what extent did national or local authorities, institutions or communities demonstrate an increased capacity and commitment to gender, disability and socially inclusive WASH? How aligned have approaches been to the needs of partner (national, provincial and/or local) governments and communities? Has it stayed relevant? What project strategies were most effective?

Equity: To what extent did the project reach and meaningfully involve the most marginalised and socially excluded people? 

Contributing to this, the evaluation will consider the four end of programme outcomes, which are aligned to the overall Water for Women Fund Theory of Change.

End of Programme Outcomes are to be evaluated against each projects agreed results. The following are indicative guidance questions from the m&e plan.

Systems Strengthening: 1 What changes have occurred in the capacity, agency and performance of governments, private sector, community-based organisations (change agents with whom the project has directly interacted), in planning, investing and delivering sustainable, gender and inclusive WASH services?

Access to WASH Services 2. To what extent have changes occurred in equitable, universal access to and use of sustainable WASH services, particularly for marginalised communities and community members?

Gender, disability and social inclusion 3. What broader (that is, beyond the domain of water, sanitation and hygiene) social inclusion changes have occurred in households, community and institutions as a result of gender and socially inclusive WASH approaches and outcomes?

Knowledge and Learning 4: In what ways have other CSOs, governments, or other organisations (within and beyond Fund participants) taken up and used Fund-generated documentation, evidence and/or demonstrated practice? How has this informed, influenced or changed policy or practice?

The evaluation will involve 4 stages – design, data collection, synthesis and results sharing. It includes both individual project level processes with in country workshops, and the overarching multi-country programme level.  These stages and their roles are described as follows. 

The following are the key indicative tasks aligned to the different stages 

5These are available online at https://snv.org/assets/explore/download/2019-1-impact-indicators-ssh4a-performance-monitoring-review.pdf and https://snv.org/assets/explore/download/2019-2-outcome-indicators-ssh4a-performance-monitoring-review.pdf

Key Roles aligned to tasks for national consultant teams

10 days per country project (Nepal)

Design Phase  

• Prepare for assignment (reading of relevant documents such as project document, baseline survey report, annual reports, etc.) 
• Participate in the 3 Hr Design workshop week 1 December
• Further define data collection tools based on consultations with country-based teams, evaluation lead and GEDSI specialist
• Develop evaluation plan, methodology and further information needed to build on existing data 

Project data collection  

• Complete data collection process in country in coordination with country teams and with oversight from evaluation lead in Dailekh and Sarlahi districts in December 2022 in Nepal (8 days).
• Complete data cleaning and provide raw data files.

Interpretation

• Support data synthesis and data analysis and collaboratively identify key findings and actions (1 day) 
• Participate in country level sense-making workshops (1/2) 

Results Sharing  

• Review country level evaluation report for consistency (1/2 day).

Key Deliverables

The main deliverables are:

• Agreed data collection plan
• Raw and clean data files.
• Short report summarising key findings against the evaluation questions, and implications for the project extension 

SNV teams will provide logistical support throughout the process and facilitate meetings with key government partners, stakeholders and partners as required. 

Timeline and Duration

The assignment is estimated to cover 10 days.

The entire assignment is anticipated to commence at the start of December with the final deliverables to be submitted to SNV on or before 31st December 2022.

Payment Terms and Conditions

The payment schedule shall be as follows:

• 25% upon signing of contract
• 75% upon completion.

The payment and contracting of country level teams will be managed by SNV in Nepal.

SNV are seeking applications for national consultants or teams with demonstrated experience in undertaking qualitative research as part of WASH programme evaluations. The consultants must have availability throughout the duration of the project and bring excellent evaluation, coordination and communication skills and a demonstrated ability to work across culturally diverse settings and teams. Experience in facilitating inclusive research processes preferred.

Composition of the national consultants team

The national consultants team is expected to be a Lead researcher but a team approach (maximum of 2) may also be appropriate with adequate steering.

The lead should meet the following criteria:

• At least 10 years of experience within the field of international development cooperation;
• A strong background in evaluation and research, including applied to WASH programming;
• Experience with qualitative research methods.
• Demonstrable familiarity with the domain of civil society strengthening, evidence generation and dissemination, and advocacy in WASH programming;
• Experience in ensuring inclusive evaluation processes and an understanding of ensuring accessibility and reasonable accommodation in evaluation processes (or similar).
• Familiarity with the sector the project is operating in;
• Fluency in English communication, reporting and presentation skills.

Evaluation Criteria 

• Experience in similar work with evidence (40%)
• Competencies (30%) 
• Financial (30%).

Due Diligence 

The consultant shall comply with the SNV Code of Conduct and the Child Protection Policy during the assignment period.

Submission of proposal

Through this request SNV invites interested parties to submit a concise proposal to SNV at email: nepal@snv.org that; 

• Expresses understanding of the purpose of the evaluation
• Provides daily consultancy fees, travel budget and an overall budget on headlines
• Includes consultants’ competencies and proof thereof (include 2 /research reports related to the subject), consultants CVs, company profile
• Declares consultants’ availability in the December 2022  
• Includes a declaration of no conflict of interests by persons and organizations involved in the evaluation.

Subject:  External evaluation of Beyond Finish Line Programme

Deadline for applications is on 02 December 2022  

Annex A  Project Summaries

Beyond the Finish Line - Inclusive and Sustainable Rural Water Supply Services in Nepal

https://snv.org/project/inclusive-and-sustainable-rural-water-supply-services-nepal

The project aims to use the opportunity of decentralization in Nepal to develop inclusive, sustainable and resilient rural water supply services and hygiene behaviour change communication for households, schools and health facilities (40,000 people, 30 institutions) in two districts as a role model for inclusive WASH services in two provinces of 6.5 million people.

Although Nepal has a high coverage of water supply (87%), only 25% of those systems are well functioning as per national monitoring indicators. The challenge is therefore, beyond the finish line, in sustaining hygiene behaviours as well as the quantity, quality, accessibility and reliability (QQAR) of water supply services. The burden of poor hygiene and water lies disproportionately with women/girls, poor, low-caste and ethnic minority groups.

Building on its experience in strengthening gravity-fed rural water supply services, SNV will engage with the new local bodies formulated in Nepal to address gender and social inclusion within WASH governance, water supply service systems, and hygiene promotion in both gravity-fed and groundwater areas.