Terms of Reference

Terms of Reference

BACKGROUND

VSO is the world’s leading independent, international development organization that works through blend of community, national and international volunteers. VSO is working in Nepal since 1964 to improve the lives of impoverished people, particularly focusing on marginalized groups, vulnerable women, and girls. We believe that people are the key to overcoming poverty, and that is why we work through Volunteering for Development Approach that people are empowered, public services are inclusive and accessible and decisions makers are held to account.

With funding from EU, Civil Society Organizations (CSOs) for a better, inclusive, and resilient education system in Nepal, VSO has been implementing SAHAYATRA project (January 2021-June 2024) in four districts (Bara, Rautahat, Parsa and Sarlahi) of Madhesh province Nepal. The focus of the project is to strengthen governance to ensure inclusive quality education for marginal community girls and children with disabilities in the context of COVID-19 and other disasters.

This project also envisions a world where even severely marginalized girls and children of disability at grade 1-5 are empowered and in the COVID 19 context through inclusive quality education to improve literacy, numeracy, and life skills. Under this approach, Big Sisters model is used to support the marginalized learners and children with disabilities bring them into classroom and retain them in classroom. In partnership with Jan Jagaran Youth Club (JJYC), National Campaign for Education Nepal (NCE-N) and Prerana, the project targets 1,600(F 900; M 700; 50 people with disabilities) marginalized people.

Major project interventions of SAHAYATRA providing mentoring support to marginalized and vulnerable children, developing capacity among the teachers and parents with regard to gender responsive education, developing CSOs capacity, strengthening capacity and supporting of the federal level, provincial and local government to develop inclusive education policy and governance. Moreover, Child Protection and Child Safeguarding are other major focus areas of the interventions.

SAHAYATRA is a four-year project started in January 2021 and scheduled to end in June 2024. The overall impact of the project is ‘Inclusive quality education for marginalized girls & children with disabilities through resilient, gender-responsive education governance enhanced by CSO’.

There is co-funding requirement for VSO for achieving outcomes of SAHAYATRA and ACTIVE is contributing to this process with increased alignment of volunteering for development.

Overall objective: Improve inclusive education for marginal girls and children with disabilities through resilient, gender responsive education governance enhanced by CSO engagement.

Specific objectives:

• To increase inclusive access to quality education and learning for marginalized girls and Children with disabilities in the Covid and post Covid
• To enhance local authorities’ accountabilities for resilience and gender responsive education governance
• To improve civil society influence and engagement in promoting right to education and advocating for resilience, gender responsive education governance.

This will be achieved through the following outcomes.

Outcome 1: To increase literacy, numeracy, and life skills for marginalized girls and Children with disabilities in Bara and Rautahat.

Outcome 2: To promote school-centered civic engagement in four districts of Madesh Province with marginalized communities collaborating to minimize educational barriers.

Outcome 3: To enhance palika capacities in engagement in four districts of Madesh Province on DRR, resilience, governance, inclusive education, and data.

Outcome 4: To strengthen CSOs capacity to contribute to good governance in education, advocacy, and DRR in Madesh Province.

Geographical Locations

VSO is currently looking for an external evaluator/consulting firm to carry out the mid-term assessment of SAHAYATRA project.

Theory of Change (ToC)

The theory of change of the project posits that the socio-cultural and political barriers that marginalized girls and boys face in primary level education contributes to their low educational access, retention, and attainment.

• At individual level, children lack a voice to advocate for their own education, and girls in particular are made to feel unsafe and unsupported in school.
• At family and community level, entrenched poverty makes it difficult for parents to prioritize their children’s education. At the same time, poor knowledge of educational rights, equity provisions (equal access to education, equal value to girls’ and boys’ education, equal time for literacy and learning at home etc.), and school management processes hinder the ability of parents to advocate for their children’s enrolment and learning. Within the community, social norms, harmful traditional practices such as early and forced child marriage, migration for labour, and expectations around gender roles keep marginalised children out of school and limit their aspirations for learning.
• At school level: Schools and school authorities struggle to materialize inclusive education policies and fail to develop teachers with child-centred, gender-responsive teaching skills.
• At CSOs level: Inadequate capacity and low engagement of CSOs in policy, planning and governance.
• At government level: Policy makers lack evidence on how to implement inclusive education policies in the new federal system and where local and provincial units require guidance and capacity building to perform their roles. Poor implementation of inclusive policies and lack of accountability by duty bearers and service providers

The successful applicant will design, plan, and conduct a mid-term evaluation of the project consisting of mixed-method, gender-Responsive evaluation, persons with disabilities and marginalized girls.

The evaluation will need to assess the delivery, effectiveness, contribution of Volunteering for Development, impact of the project and recommend the findings and promising practices of the project interventions on marginalized students' education and their lives. The findings from the study will primarily be used:

a) By the project management team, project partners and stakeholders to inform improvements in the delivery of the project during its lifetime.

b) To demonstrate accountability for the funding received from the EU.

c) By the project management team to leverage additional resources from existing and new partners and stakeholders to scale-up and sustain the activities/benefits delivered by the project.

d) By the project management team to support the on-going development and implementation of the project’s sustainability and succession strategies.

SCOPE OF WORK:

These below are the principles underpinning the study:

• Results in learning that is useful not just for partners, EU, and VSO but the education sector more broadly.
• Builds on the priority learning areas.
• Solicits input at key points from all the stakeholders.

Evaluation Questions

The applicant is required to develop an evaluation approach that answers the following overarching questions:

1. What is the project’s impact on the learning outcomes of marginalized girls and children with disabilities and their successful transition from class 5 to 6?
2. What works to create an inclusive, child friendly, gender responsive learning environment at home, community, and school to support marginalized girls and children with disabilities from class 1-5?
3. How far the project is able to strengthen disaster preparedness, resilience, disability-sensitive and gender-responsive education governance?
4. How far the project has been able to support active citizenship and accountability through CSOs?

These questions help define the scope and focus of the project evaluation process. The successful bidder will be expected to work with the Project Team to review and revise these questions as appropriate at the outset of the project.

The research will be carried out through field visits in Bara and Rautahat districts and through virtual mode in Parsa and Sarlahi districts.

DELIVERABLES & TIMELINES (See example below)

Work plan: Provide a detailed work plan incorporating all relevant tasks and milestones from start to completion of the evaluation study. These are the main deliverables for the bidders:

• Inception report: setting out the research design and associated planning, logistics, quality assurance, child protection measures, Do No Harm, and risk management information including gender equality and social inclusion analysis.
• Mid-line project evaluation report: Design, conduct and submit a mid-term assessment report and updated log frame that describes the conditions of SAHAYATRA project indicators against the baseline value which progress can be measured, or comparisons can be made to show the effects and impacts of the project.
• Data: All the raw data set that has been collected during mid-term assessment.
• Stories (Case studies/ Success Stories): Generate 3 case stories including one for ACTIVE Citizens in English Language.

Project milestones: Please include the dates for the project milestones outlined in the work plan below:

Note: Detailed steps, access, need to be designed and their implementation timeline need to be mentioned clearly on work plan.

Reporting requirements: Please confirm you will follow our reporting requirements outlined below.

• All reports along with a standard PPT presentation should be submitted in both hard (3 copies) and soft copies and should be submitted in English in the specified timeframe.
• The Bidder will be required to provide face-to-face or distance presentations of all deliverables as an integral part of the submission process.
• The Bidder will be expected to provide a fully ‘cleaned-up’ dataset in SPSS, Excel Spread sheet, Stata or SAS file format accompanied by the code used to carry out analysis and a variable codebook. The bidder will also be expected to submit qualitative transcripts.

Contracting Arrangements: Please confirm you will follow our contracting requirements outlined below:

• The Bidder will be expected to identify a Project Director and Project Manager for communication and reporting purposes. At the Inception meeting the Bidder Project Manager will be expected to submit a full contact list of all those involved in the evaluation.
• The Bidder will be expected to report to the Evaluation Steering Group and attend all meetings as agreed with the Project Manager. The Team will be required to submit the bi-weekly progress reports (by email) to the Project Manager during the study periods summarizing activities /tasks completed to date (percent achieved), time spent etc.

METHODOLOGY:

Research Design: Please describe in detail what research design you intend for the midterm study to follow for evaluating the impact of the project. Please explain why this research design is best suited for SAHAYATRA project and how it links to our evaluation objectives and requirements. The design will also be guided by Government of Nepal’s Early Grade Reading Assessment (EGRA) and Early Grade Mathematical Assessment (EGMA) to understand literacy and numeracy-related skills of students that project targets in the working area.

Sampling: Please describe how you will work with the VSO to finalize the sampling frameworks for both qualitative and quantitative samples. Please also propose qualitative and quantitative sample to set midline targets for project outcomes and indicators which has sufficient size and representatives to allow for reasonable:

• Levels of certainty that the findings are representative for the target population.
• Ability to generalize the intervention’s effectiveness to similar contexts.
• Ability to generalize the insights into what works and why for similar contexts.

Measuring outcomes: Please describe how you will use quantitative and qualitative approaches to measure the outcomes and outputs for this project.

Tool development: Please describe what quantitative and qualitative tools are required for this evaluation and how you will develop them to ensure they are robust and fit for the purpose.

Log frame development: Please describe how you will work with a project to refine what qualitative and quantitative indicators are in the log frame to help ensure if the evaluation achieves its objectives.

Data collection (method): Please state which party will conduct the data collection and how they will collect both qualitative and quantitative data. How do you plan to collect data using both face to face and virtual methods? For example, will software be used? Do you intend to sequence the data collection process between quantitative and qualitative information?

Data collection (skills and experience): Please also describe how they have the necessary skills and experience to collect robust qualitative and quantitative data for education projects in Nepal.

Data cleaning: Please describe your approach for how qualitative and quantitative data will be cleaned to ensure it is robust.

Gender and disability sensitive: Please describe what steps you will take to ensure this evaluation is gender and disability sensitive.

Data analysis: Please describe your approach to conducting a mixed method data analysis (i.e., data triangulation).

Reporting your findings and recommendations: Please describe your approach to reporting your findings and providing recommendations.

Ethical Protocols: As this evaluation must consider the safety of participants and especially children at all stages of the evaluation, please demonstrate your approach to protection and safeguarding of children through the different evaluation stages, including recruitment and training of research staff, data collection and data analysis and report writing. Please note, at a minimum, your proposed approach should comply with international good practice with regards to research ethics and protocols particularly with regards to safeguarding children, vulnerable groups (including people with disabilities) and those in fragile and conflict affected states. Consideration should be given to:

• Administrative, technical, and physical safeguards to protect the confidentiality of those participating in the research.
• Physical safeguards for those conducting research.
• Data protection and secure maintenance procedures for personal information that needs to be informed by GDPR 2018.
• Parental consent concerning data collection or collation of data about children under 18.
• Age and ability appropriate instructions based on reasonable assumptions about comprehension for the ages of children and the disabilities they intend to involve in the research; and
• Age-appropriate participation of children, including in the development of data collection tools.

Risk Management: Please submit a comprehensive risk management plan covering:

• The assumptions underpinning the successful completion of the proposals submitted and the anticipated challenges that might be faced.
• Estimates of the level of risk for each risk identified.
• Proposed contingency plans that the bidder will put in place to mitigate against any occurrence of each of the identified risk.
• Specific child protection risks and mitigating strategies, including reference to the child protection policy and procedures that will be in place; and
• Health and safety issues that may require significant duty of care precautions.

Quality Assurance Plan: Please submit a quality assurance plan that sets out the systems and processes for quality assuring the evaluation and research process and deliverables from start to the completion of the project. This plan should include the proposed approaches for:

• Piloting of all research activities.
• Training of enumerators and researchers conducting mixed-methods primary research, including in research ethics.
• Logistic and management planning.
• Field work and virtual interaction protocols and data verification including back-checking and quality control by supervisors; and
• Data cleaning and editing before any analysis.

SKILL, KNOWLEDGE, & EXPERIENCE

Relevant experience: Please describe experience of your consulting firm and team involved on conducting evaluations of education projects. Please be specific in stating whether the work mentioned involved the following:

• Evaluation of education projects
• Conducting longitudinal evaluations
• Conducting mixed-method evaluations
• Disability and gender responsiveness and sensitivity

Profession Skills and Qualifications: Please provide CVs Maximum 3 pages as per the template attached as Annex 1 of all those proposed in the Bid, clearly stating their roles and responsibilities for conducting evaluations. Please note that if the enumeration is to be subcontracted, the evaluator will be ultimately responsible for the enumeration they are subcontracting to this task. The proposed evaluation team should include the technical expertise and practical experience required to deliver the scope of work and evaluation outputs in particular.

• Evaluation design: The team should include skills and expertise required to design, plan, and conduct mixed-method impact evaluation, potentially using experimental or quasi-experimental techniques.
• Qualitative and quantitative: Skills in quantitative and qualitative data collection and analysis, drawing findings from multiple sources and handling potential contradictions between data sets. It is also crucial to demonstrate skills and experience in triangulating quantitative and qualitative data.
• Relevant subject matter knowledge and experience: Knowledge and experience required on conducting research with children, the education sector, disability, and gender to ensure that the evaluation design and research methods are as relevant and meaningful as possible given the aims and objectives of the project and the context in which it is being delivered.
• Evaluation management: Manage a potentially large-scale and complex evaluation and research process from end-to-end, including conducting and reporting a baseline study and final project evaluation report.
• Primary research: Gender-sensitive design, management, and implementation of primary quantitative and qualitative research in potentially challenging project environments, such as fragile and conflict affected states – this could include the design of longitudinal household panel surveys, EGRA /EGMA tests, in-depth interviews, focus groups, etc.
• Learning tests: Designing and administering learning tests
• Country experience: It is particularly important that the team has the appropriate country knowledge/experience and language proficiency required to conduct the assessment as required.
• Information management: Design and manage disaggregated data and information systems including sex, age, ethnicity, and disability. Capable of handling large datasets for MEL purposes.
• Statistical analysis: A range of statistical modelling and analysis of impact data, highly proficient user of: SPSS or STATA; and qualitative data analysis techniques, including the use of software e.g., ATLAS.ti, NVivo or equivalent where needed.
• VfM assessment of education projects: Education economics expertise to conduct cost benefit analysis and cost effectiveness analysis as a part of the assessment of the project’s VfM; and
• Safety considerations: Ensuring the whole evaluation process adheres to best practice for research with children including the implementation of child protection policy and procedures to ensure safety of participants. Note that all bidders are expected to be able to show that they have a child protection policy in place to safeguard children that the research team would come into contact with through the research activities.

References: Please provide two references who we may contact to discuss experiences of working with you.

Payment

Payment will be made in installments:

1. First Installment: 30% after submission of the work plan and inception report in agreement with VSO.
2. Second Installment: 30% after submission of the draft report satisfactory to VSO.
3. Third Installment: 40% after submission of the final report satisfactory to VSO.

The budget should be inclusive of all costs covering team member costs, travel, research costs and any other costs associated with the completion of work including the miscellaneous costs. Bidders are required to organize and fund their own duty of care arrangements as required. Please provide a fully costed proposal in the form of a price schedule that as a minimum should include:

1. Sub-total of fees for the delivery of any task or deliverable.
2. Sub-total for number of days per district (as applicable).
3. Expenses and overheads broken down by the project cost categories [Recipient to provide].
4. Reasonable adjustment costs; and
5. Total costs before and after any taxes that are applicable.

Application

The following are the key instructions for bidders/Consulting firms who are interested in submitting a proposal.

• While submitting this bid, the evaluator should submit a valid company registration certificate, PAN/VAT registration certificate, tax clearance certificate of FY 2078/79, organizational profile, structure of governing bodies, and areas of expertise.
• The length of the bidding document should be approximately 15-20 pages.
• Please use Calibri font size 11.
• Interested firms are invited to submit their application including proposed daily professional rate, a summary of similar assignment/ research / study / assessment conducted and sample of at least one assessment report completed recently.
• The bid document should be submitted to the VSO by 5:00 PM at procurement.nepal@vsoint.org, Nepal time, 17 September 2023. Proposals submitted after 5:00 PM on that date will not be taken into consideration.
• The value of the Technical and Financial proposals will be evaluated at a ratio of 70:30 respectively.

Annex 1 Template_MID TERM ASSESSMENT