Terms of Reference (ToR) for External Evaluation

Terms of Reference (ToR) for

External Evaluation

Rautahat Socio-Economic Recovery Program

1. Background

Islamic Relief Worldwide (IRW) is an independent humanitarian and development organization that has been serving humanity for more than 38 years. With an active presence in over 45 countries across the globe, IR strives to make the world a better and fairer place for the three billion people still living in poverty. Since 1984, IRW has helped millions of the world’s poorest and most vulnerable people. Inspired by the Islamic faith and guided by Islamic values, IRW believes that people with wealth have a duty to those less fortunate – regardless of race, political affiliation, gender, or belief. IRW is affiliated with Social Welfare Council as an International Non-Government Organization.

IRW projects provide poor people with access to vital services and protect communities, women and children from/during disasters and deliver life-saving emergency aid. It provides lasting routes out of poverty and empowers vulnerable people to transform their lives and their communities. IRW supported local partners immediately after the Gorkha Earthquake 2015 to address the Emergency and recovery needs of the most affected population. IRW is presently working through local implementing partners in Rautahat, Rasuwa, Kapilvastu, and Rupandehi districts.

Islamic Relief (IR) Nepal has been implementing Livelihood, WASH and DRR integrated intervention in the Rautahat district through a local implementing partner- Rural Development Centre (RDC) Nepal. This intervention: Rautahat Socio-Economic Recovery program was launched in January 2021 and targets two municipalities (Paroha and Ishnath) of the Rautahat district.

The project aims to “improve socio-economic resilience of vulnerable HHs to rebuild lives that is interdependent on sustainable livelihood, diversified income, improved access to WASH and reduced disaster risk profile” for 5,055 direct beneficiaries in Paroha and Ishnath municipalities.  The proposed project is aligned with IRW’s Socio-Economic Recovery Framework (SERF). The overall approach highlights a needs-based, system-oriented, and protection-sensitive initiative that visualizes the continued commitment towards the organizational mandates specifically on “Do no harm” and “Leaving no one behind”. The project focuses primarily on those who are most vulnerable and heavily impacted by livelihood loss, inadequate access to WASH, and not experienced good practices in disaster preparedness and risk reduction interventions.

IR considers evaluation as an integral part of the implementation of interventions on the ground. This project will go through independent evaluations (Endline and End of Project evaluations). Thus, this TOR sets out the responsibilities of the consultant to design and carry out project endline and end-of-project evaluation in different communities where project activities are implemented in the Rautahat district. The evaluation will focus on the assessment of the implemented activities and whether the activities lead to the achievement of the planned results as well as the project objectives. As a result of this evaluation, recommendations from the evaluator/s are expected to improve the quality of ongoing and future projects.

2. Summary Table

3. Scope of Evaluation 

The objective of the current Terms of Reference (ToR) is to deliver the Endline study and End of Project Evaluation of the project “Rautahat Socio-Economic Recovery Program” implemented by RDC as a local partner.

The specific objectives of this endline study are to:

• To establish a final value for the project results against the project target in the project area. 
• To create an endline value for measuring the outcomes of the project
• Document at least 5 case studies and 2 Most Significant Change stories (Indicator Impact Story) of the project demonstrating project impact

The evaluation will apply the DAC criteria: relevance, effectiveness, efficiency, impact, Timeliness, and sustainability.

• Evaluate the project’s theory of change (ToC); its relevance to the overall country strategy; the appropriateness of its planned objectives in terms of funding and duration of the intervention; and its appropriateness of project design, intervention, and needs of the beneficiaries.  
• To assess the effectiveness and relevance of projects interventions, outputs, and results
• To assess the key effects /outcomes of the project; Provide evidence of the achievements of this project in terms of both the positive and negative, intended, and unintended, and the primary and secondary effects of the program, alongside any direct or indirect contributions to any systemic change.
• Assess the key innovations used in the project and their impact, whether positive or negative, upon the delivery of project deliverables.
• To assess the effectiveness of accountability systems and the functionality of complaint response mechanisms in the targeted communities
• Document lessons learned and developed clear and actionable recommendations for adoption and integration into any similar future development-related projects within the region and elsewhere.
• To identify the Most Significant Change (MSC) of the

4. Focus of the Evaluation

The end of project external evaluation aligning to DAC criteria should broadly consider but not be limited to the following questions to discuss, provide conclusions and recommendations.

Relevance: (Assess the design and Focus of the Project)

• Was the project designed in a way that is relevant to reaching its goals?
• To what extent the implemented activities were relevant to the need of the people targeted by the project?
• Was the project relevant in terms of context, urgency, timeframe, local and national realities and priorities and resources used by the project to achieve the project goal?

Effectiveness: (Processes and their appropriateness in supporting project delivery)

• To which degree did the activities meet the objectives and results set out in the project (as outlined in the logical framework)?
• What are the intended and unintended, primary, and secondary effects produced by the intervention?
• How useful is the Farmer’s group established and functional at the community level?
• Did the project/activities meet the relevant needs of the rightsholders to address the socio-economic issues? and what was the result?
• To what extent that project monitoring and evaluation mechanisms contribute to achieving the project result? 
• How effective were the tools and strategies used in the implementation of the project?
• What are the major factors influencing the achievement or non-achievement of outcomes?

Efficiency: (Project Implementation)

• Was the project run in an efficient way?
• Was the process to achieve results was efficient? Were the resources effectively utilized?
• Did the project justify value for money with each activity?
• Were the project activities carried out in a timely manner and objectives achieved on time?
• Were there any factors that reduced/enhanced the efficiency during implementation?
• How can the project of similar nature enhance its efficiency to meet its objective? 
• Could different approaches to project implementation have produced better results?
• What was the strength, weakness, opportunities, and threats of the project implementation process?

Timeliness:

• Were the project design and interventions timely in responding to the needs on the ground?
• Were, the activities timely implemented in comparison to project planning?
• Were funds available in time during the implementation of the activities to respond to new developments,
• To what extent did the collaboration between Islamic Relief Nepal and local government (Municipalities and wards) and other stakeholders contribute to the efficient and timely coordination of activities and processes?

Sustainability: 

• What is the extent that the benefits of the projects are likely to be sustained after completion of the project? Are the positive effects and impacts sustainable?
• Will the changes caused by the project continue beyond the life of the program and how?
• What are the potential opportunities and cautions for replicating the project in other communities?
• How effective were the exit strategies and approaches to phase out assistance by the project?
• What are the factors that contributed to or hindered in the sustainability of the project actions?

Impact: 

• What has happened because of the project?
• What real difference has the activity made to the beneficiaries?
• How has the project contributed to improving the Socio-Economic resiliency in the Paroha and Ishnath municipalities of Rautahat?

Additional specific criteria should also be assessed using the following key questions:

Gender and disability considerations:

• How has the project considered gender aspects both in the project design and its implementation of activities? 
• How has the project considered disability issues both in the project design and its implementation of activities?

Core Humanitarian Standard Commitments:

• Are the staffs supported to do their job effectively, and are treated fairly and equitably?
• Has the project learned, innovate, and implement changes based on monitoring and evaluation, and feedback and complaints received?
• Are the complaints and safeguarding issues are responded to in a timely, fair, and appropriate manner that prioritizes the safety of the complainant and those affected at all stages?

5. Methodology of the Evaluation (Endline Study and End of Project Evaluation)

The endline and end of the project evaluation study should employ both qualitative and quantitative approaches. A review of relevant secondary data shall be an integral part of the study and should be reflected wherever possible in the report. The study design should align with the project log frame. This study must measure and establish an endline value against project indicators.

The consultant will undertake a comprehensive desk review of the existing project literature and conduct both a desk study and a field visit. The literature review will include but not be limited to the project proposal, logical framework, interim/periodic reports, light KAP database, case studies, beneficiary’s database, and other relevant documents and reports prepared by IRW- Nepal and its partner. The IRW- Nepal will provide all relevant key project documents for the comprehensive desk review.

Based on the literature review the consultant will develop the methodology for primary data collection for both the endline study and end-of-project evaluation. For primary data collection, the consultant will engage in the field and collect data from project-targeted areas as specified in the indicative schedule.

6. Deliverables

All deliverables are to be submitted to Islamic Relief Nepal in both electronic and hard copy format. Deliverables include:

• An inception report, to be submitted one week after the agreement, explaining the methodologies, data collection and reporting plans with draft data collection tools such as interview (KII), FGD guides, a timeframe with fixed dates for deliverables for both endline study and end of project evaluation. 
• A final report (combined or separate) to be submitted at the end of the evaluation with a maximum extension of 30 A4 size pages excluding annexes.

7. Indicative Schedule 

The consultancy will last 4 weeks, and it is expected to be carried out during 1 December – 28 December 2022 (tentative). The selection and contractual procedures will complete by end of November 2022. The tentative schedule is as follows:

• Week 1: Desk review of core documents; drafting and validation of the inception report with detailed work plan, checklists, tools, and questionnaires. Enumerators’ orientation after the approval of tools and inception report.
• Weeks 2-3: Field visits to project sites; interviews with project staff, beneficiaries, and relevant stakeholders. Focus group discussions (FGDs) with members of farmers group, WASH group, etc
• Weeks 3-4: Drafting of the final report (final version to be submitted no later than 30 December 2022).

8. Quality and Ethical Standards 

During the evaluation, it should take all reasonable steps to ensure evaluation is designed and conducted to respect and protect the dignity, rights and welfare of the people, communities, stakeholders, and authorities involved and to ensure that the evaluation is technically accurate and reliable, is conducted transparently and impartially, and with full accountability.

The consultant will sign and adhere to the Islamic Relief’s safeguarding and protection against exploitation and sexual abuse (PESA) code of conduct.

The study standards are:

• Utility: The evaluation product must be useful as an organizational learning. 
• Feasibility: Evaluation must be realistic, diplomatic, and managed in a sensible, cost-effective manner.
• Ethics & Legality: Evaluation must be conducted ethically and legally, with regard for the welfare of those involved in and affected by the evaluation.
• Impartiality: Evaluation should be impartial, providing a comprehensive and unbiased assessment that considers the views of all stakeholders
• Transparency: Evaluation activities should reflect an attitude of openness and transparency.
• Accuracy: Evaluation should be technically accurate, providing enough information, analysis, and interpretation.
• Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate.
• Collaboration: During the evaluation, collaboration at different levels is expected. This will later improve the legitimacy and utility of the study.
• Consent: Taking formal consent of all the responders is must during the evaluation.

9. Evaluation Team

The Evaluation Team is expected to:

• Be composed of a team leader with documented extensive experience on similar evaluations of development projects in the field of WASH, DRR, livelihood and Protection.
• Have extensive experience in conducting endline study and external evaluations in the context of cooperation for development and a proven record of delivering professional results.
• Have sound knowledge of evaluation and data-collection methods.
• Be able to communicate effectively in Nepali and English.
• Have previous working experience in Terai (Madesh Province).

10. Proposal Evaluation 

The technical and financial proposals will be evaluated separately and have 70% and 30% scores respectively. The proposals evaluation is based on the given criteria

11. Application Procedure

Interested candidates, firms and agencies are mandatory to submit the following documents by 27 November 2022 with the subject REF: Endline and End of Project Evaluation-Recover to Info.Nepal@irworldwide.org.

i) Letter of motivation (7 pages max.) indicating

• The consultants’ suitability for the assignment and match with previous work experience, qualifications etc.
• How the team will be composed and the division of work between team members.
• Discussion of the work methodology it will use.
• Draft work plan and suggested timetable.
• Economic offer and budget break down.

ii) Professional profile of the evaluating team/ company (CV of least lead consultant/team leader of consultancy team)

iii) A copy of the previous evaluation report on the similar issue

For firms/agencies following additional documents must be submitted:

• Valid certificate of company registration
• Valid copy of tax registration certificate
• Tax clearance certificate

Islamic Relief Worldwide, Country Office Nepal

House no. 482, Kumaripati-5 (Near United Academy)

Lalitpur Metropolitan City, Nepal

Contact no: +977 1 5537610