IDF TB Project Evaluation International Consultant, IOM Moldova

CALL FOR APPLICATIONS   

Reference code:  SVN 2024-11

Established in 1951, IOM is a Related Organization of the United Nations, and as the leading UN agency in the field of migration, works closely with governmental, intergovernmental and non-governmental partners. IOM is dedicated to promoting humane and orderly migration for the benefit of all. It does so by providing services and advice to governments and migrants.

1. Evaluation context

The International Organization for Migration (IOM), part of the UN system as a related organization, is the leading inter-governmental organization in the field of migration and works closely with governmental, intergovernmental, and non-governmental partners. With 172 Member States, a further eight states holding observer status, and offices in over 100 countries, IOM is dedicated to promoting humane and orderly migration for the benefit of all. It does so by providing services and advice to governments and migrants. 

IOM has long-standing experience advocating for migrants’ right to health services and implementing migration health programming globally. IOM Moldova has been conducting migration health assessments for over 20 years and has been implementing health-related promotion activities targeting migrants and refugees for several years, including gender-sensitive approaches. The IOM Mission in the Republic of Moldova implemented the project “Enhancing Gender-sensitive TB Detention, Surveillance, Treatment and Prevention among Mobile Populations from the Republic of Moldova” (MA.0494). The project’s main goal was to contribute to good migration and health governance in the Republic of Moldova by developing effective migrant-centered and evidence-based policies, and mainstreaming gender and migrant-sensitive Tuberculosis (TB) interventions throughout the Moldovan National TB Programme (NTP) while contributing to empowering TB-affected migrants and their families as development actors. The coordination and partnership between IOM, NTP, the Phthisiopneumology Institute “Chiril Draganiuc” (PPI), and the Ministry of Health (MoH) were established to ensure the implementation of the project activities. 

2. Evaluation purpose 

This evaluation is being conducted by the evaluation guidelines of the donor, the IOM Development Fund (the Fund) which calls for an ex-post evaluation of the project to be carried out 12 months after the end of the project. The overall objective is to assess the achieved results highlighting strengths and weaknesses in project design, management, and implementation (process evaluation); how successful the project was in producing desired results (outcome evaluation); as well as touching upon produced impact and sustainability of project's benefits.

The evaluation will be used primarily by IOM and the Fund to assess performance, effectiveness, and sustainability and to inform future decision-making on the use of the Fund. The evaluation was designed to be conducted 12 months after the end of the project to allow time for the outcomes of the project to materialize and for lessons learned within the project to be analyzed for eventual incorporation into policy and program-making of the concerned institutions. 

The evaluation is being conducted for practical use by: a) the project management team to improve the implementation of the activities under the IOM Moldova Mission’s Migration Health portfolio; b) the Fund, to get familiar with good practices, evaluate the project stakeholder satisfaction with the project activities and results and accordingly assess value for money for the results it has funded; and c) IOM staff supporting similar projects by sharing lessons learned and good practices. 

3. Evaluation scope

The evaluation will cover the entire period of project implementation (1 November 2020 to 31 October 2022) of the “Enhancing Gender-sensitive TB Detention, Surveillance, Treatment and Prevention among Mobile Populations from the Republic of Moldova” project up to the time of the evaluation. The evaluation will take place mainly in Chisinau, Republic of Moldova, and will also potentially include telephone/Microsoft Teams interviews and/or written surveys of beneficiaries/stakeholders.

4. Evaluation criteria

The evaluation will be conducted in line with the IOM Development Fund Evaluation Guidelines and will use the six main OECD-DAC criteria of relevance, coherence, effectiveness, efficiency, impact, and sustainability as well as cross-cutting issues of gender and human rights. Also, in terms of gender, all findings will be disaggregated by sex (as relevant and to the extent possible) and, analyzed and reported on accordingly. 

5. Evaluation questions

The following key questions will guide the evaluation:

Relevance

• Do the intended results align with and support government policies and national strategies?
• Has the project responded to the needs of the target beneficiaries, both stakeholders, health providers, and migrant TB patients/beneficiaries?
• Are there any identifiable ways that the approach should be revised in the future, or is it evident that additional or complementary activities or projects will need to be implemented?
• Is the project document well designed according to IOM project development guidelines and logical framework approach, including results, indicators, budget, and planning?

Coherence

• How does this project align with and complement other related initiatives, whether implemented by IOM, the government, or other national and international actors? What is the added value, if any, of this project compared to those other efforts?
• What other synergies and linkages with authorities, partners, and other stakeholders could be made in future interventions?

Effectiveness

• Have the project outputs and outcomes been achieved in accordance with the stated plans, or are they on track to being achieved? If not, why not?
• Were the activities sufficiently well implemented to reach the intended results? Would other activities have been more effective in reaching the results?
• To what extent has the project adapted to changing external conditions to ensure project outcomes are achieved?

What were the major external factors influencing the achievement of the project’s expected outputs and outcomes, including both contextual factors and other related interventions?

Efficiency

• To what extent were resources (time, funds, expertise) adequate to implement the activities and achieve the outputs?
• Were the project activities undertaken as scheduled and were outputs delivered on time? If not, what was the reason for possible delays?
• What measures have been taken to ensure that resources are efficiently used?

Impact

• To what extent can long-term changes be observed (whether intended or unintended, positive, or negative), particularly in relation to building capacities of relevant State and nonstate actors?
• What are the direct and indirect impacts of the project at the outcome level?

Sustainability

• Are the benefits generated likely to continue once external support ceases?
• To what extent were the project and its results supported by local institutions and embedded in institutional structures that are surviving beyond the life of the project? In particular:
  • To what extent the SIME TB Migration Module is used and maintained by the Phthisiopneumology Institute?
  • How the research and training materials have been taken up and used after the end of the project by stakeholders and beneficiaries?
• What are the major factors affecting sustainability, including any identified challenges faced by the implementing organization, partner organizations, and stakeholders?

Cross-cutting issues

• To what extent were gender mainstreaming issues considered in design and implementation?
• Were any barriers to equal gender participation identified in design or implementation, and was anything done to address these barriers? Was the planning and design process for the project participatory and non-discriminatory?
• To what extent has the project helped to ensure respect for the relevant human rights of migrants?

6. Evaluation methodology 

An evaluation framework will be developed and focus on all six standard OECD-DAC criteria and crosscutting themes criteria (gender, human rights), supported by standard tools (evaluation matrix).

The methodology will involve a combination of desk review and in-depth interviews to gather and triangulate data from beneficiary and partner perceptions with project data (secondary quantitative data). Specifically, the following methodology is proposed, to be revised as needed during the inception phase:

• Review of project documents (proposal, reports, work plans, monitoring data, budget, financial report, annexes, etc.); relevant IOM strategies and guidance; relevant national strategies and action plans (home-based, desk study);
• Key in-depth informant interviews with the IOM project manager, project partners, and other stakeholders as needed (fieldwork within Moldova; and online interviews, home-based);
• Collect and analyze key findings, make informed and analysis-based conclusions including lessons learned, and develop recommendations (home-based).
• Produce final evaluation deliverables in the English language (home-based).

The monitoring data to be provided to the Evaluator will include information on baselines and progress against indicators in the results matrix, feedback from stakeholders and beneficiaries involved in project implementation on the extent to which the project met expectations and enhanced their work.

The Evaluator will execute the evaluation with the support and oversight of the Project Manager, who will serve as the Evaluation Manager to support and oversee the evaluation process. The Evaluation Manager will coordinate with a focal person in the IOM Moldova to support the Evaluator in providing country-specific information, relevant documents, and information on follow-up actions that illustrate the achievements of the project. IOM Moldova will also support the logistical and administrative arrangements, including helping to organize meetings and arranging interpreters, as needed.

The Evaluator will be responsible for preparing and carrying out data collection and analysis and producing the evaluation deliverables outlined below. The Evaluator should provide periodic feedback as needed to the Evaluation Manager on progress and any challenges faced. The Evaluation Manager will be responsible for managing the process including feedback and comments to the inception report and draft evaluation report and helping to address any issues or challenges flagged by the evaluator.

The Evaluator will create an Inception Report that will provide more details on the methodologies to be used, including an evaluation matrix, initial drafts of interview guides, and an agenda for interviews and field visits (to be coordinated with IOM Moldova). The evaluation data should be disaggregated to the extent possible by gender. 

7. Ethics, norms, and standards for evaluation

The Evaluator must follow the IOM Data Protection Principles, UNEG norms and standards for evaluation, and relevant ethical guidelines. 

8. Evaluation deliverables

The deliverables expected from the Evaluator include the following:

• An inception report will be prepared within three weeks after the start of the assignment by the Evaluator and shared with IOM Moldova. The report should include a proposed timeline, an evaluation matrix, and a draft interview guide, along with any other needed data collection tools. The Evaluation Matrix will demonstrate the Evaluators’ understanding of the ToR and outline data collection and analysis plans, to be completed and reviewed with the Project Manager prior to the field visit.

• Following the field visit, the Evaluator will prepare a short presentation of the initial findings and tentative conclusions and recommendations. This will be used to debrief the Evaluation Manager and other relevant parties, to identify and address any misinterpretations or gaps. 

• Building on the debrief and initial feedback received, the Evaluator will produce a draft report that will be shared with the Evaluation Manager for review within two months after completion of fieldwork. The Evaluation Manager will request and consolidate feedback (including from the Chief of Mission for IOM Moldova, the MHD RTS at RO Vienna, and the Fund) and present it to the Evaluator. Feedback should focus on technical aspects and not on the conclusions or findings unless those are based on inaccurate or incomplete information, in which case corrected or supplemental information should be provided.

Once feedback is provided by the Evaluation Manager, the Evaluator will prepare the final report. The report will follow the same presentation logic and include, at a minimum, the information described in the IOM Project Handbook template for evaluation reports: executive summary, list of acronyms, introduction, evaluation context and purpose, evaluation framework and methodology, findings, conclusions, and recommendations. Annexes should include the TOR, inception report, list of documents reviewed, list of persons interviewed or consulted, and data collection instruments. The minimum report content requirement is as follows:

• Cover page, list of acronyms
• Table of contents
• Executive summary
• Methodology
• Findings
• Conclusions
• Recommendations in order of priority
• Annexes (itinerary, people met, question guides, etc.)
• The Evaluator will also prepare a two-page Evaluation Brief to facilitate sharing of the key findings, conclusions, and recommendations.
• The Evaluation Brief will include identification of the audience of the learning brief; project information (project title, countries covered, project type and code, project duration, project period, donor(s), and budget); evaluation background (evaluation purpose, evaluation team, evaluation timeframe, type of evaluation, methodology); and evaluation results: key findings and/or conclusions, including best practices and lessons learned, and key recommendations.
• Management response that will indicate if management accepts, partially accepts, or rejects a recommendation along with a related explanation.

All deliverables are to be written in English and meet good language standards. The final report should meet the standards laid out in the UNEG Quality Checklist for Evaluation Reports. 

9. Evaluation work plan

Based on the TOR, the Evaluator will review documents provided by the Evaluation Manager and draft the Inception Report for review by the Evaluation Manager. The Evaluation Manager will arrange an initial management meeting (short phone/online call) to ensure a common understanding of the evaluation process, confirm final logistics, and review and give inputs into data collection plans as outlined in the Inception Report. 

Field visit logistics will be the responsibility of the Evaluation Manager, including arranging meetings, transportation, and external interpretation services for meetings and field visits. An informal debrief should be included at the end of the field visit, with a summary debrief presentation provided. The presentation of the initial findings following the conclusion of the data collection and preliminary analysis will allow for any obvious oversights, misinterpretations, or information gaps to be identified and addressed before the evaluator begins drafting the full report. A draft report will then be proposed within two months of the field visit, and then finalized based on comments from the Evaluation Manager along with a brief. 

Once approved by the Evaluation Manager, the report and brief will be shared with the Fund (by the Evaluation Manager) and with other MHD specialists as needed (by the MHD Regional Thematic Specialist in RO Vienna). 

The evaluation visit is planned to take place in end-January-beginning February 2024. A precise timeline will be established with the selected Evaluator, and will consist of the following stages (dates are tentative and can be adjusted):

1. Planning and Desk Research Phase

In the desk research phase, the relevant project documents should be reviewed: The project proposal, budget, relevant guidelines, phase project, and final reports with the annexes. 

Deliverable: The Evaluator will prepare a detailed evaluation matrix with an indicative list of people to be interviewed and the methodological tools to be used. 

2. Field Phase

The Evaluator will carry out the assessment (4 working days in Moldova). The data will be collected according to the following methods and the Evaluation sheet will serve to compile the data:

• Briefing meetings with project management staff at the beginning and the end of the field phase.
• In-depth interviews with the key project partners and relevant stakeholders and beneficiaries (face to face and via telephone/Teams/skype)
• Focus group discussion with partners, and IOM staff supporting similar projects.

Deliverable: Following the field visit, the Evaluator will prepare a short presentation of the initial findings and tentative conclusions and recommendations, delivering it over a de-briefing meeting at the end of the field phase.   

3. Synthesis Phase. This phase is devoted to the development and submission of the evaluation report and two-page brief.
4. Deliverable: The Evaluator will submit a Draft Evaluation Report (within two months after the visit). Based on comments on the draft report, the Evaluator will submit the final report along with the two-page brief (within three months after the visit) including the Management Response Matrix

The estimated workload and responsibilities’ allocation read as follows:

Requirements

Interested consultants/ should meet the following requirements:

• Master’s degree in public health, policy, development studies, international relations, development economics, or any other relevant field.
• At least 10 years of professional experience in project/program evaluation.
• Experience in conducting participatory evaluations, gender and/or rights responsive evaluations, with practical experience in remote evaluation methods.
• Experience in conducting evaluations in the region, in humanitarian settings, and/or in a multiplicity of migration topics or sectors such as migration health, border management, migration, and development, etc.
• Technical competencies in real-time evaluation design, data collection, data analysis (qualitative and/or quantitative), drafting and editing, computer skills (including, if necessary, software like SPSS, STATA, etc.), communication skills, time management skills, and cultural sensitivity.
• Familiarity with the development contexts of the Republic of Moldova a distinct advantage
• Ability to work under pressure and meet deadlines
• Strong analytical and report-writing skills and abilities to present findings and practical recommendations
• Excellent written and verbal communication skills in English

Other requirements

Inclusion and respect for diversity respect and promote individual and cultural differences; encourage diversity and inclusion wherever possible.

Integrity and transparency maintain high ethical standards and act in a manner consistent with organizational principles/rules and standards of conduct.

Professionalism: demonstrates ability to work in a composed, competent, and committed manner and exercises careful judgment in meeting day-to-day challenges.

The evaluator must adhere to the IOM Data Protection Principles (IN/138).

Payment

▪ 40 % upon approval of the Inception Report

▪ 60 % upon approval of the final evaluation report and submission of all relevant documents

Apart from consultancy fees, IOM shall cover the travel costs to conduct fieldwork as appropriate.

How to apply:

Submission of application/expression of interest

Interested applicants/firms are invited to submit the following documents in English to hrchisinau@iom.int :

▪ A technical proposal that summarizes the proposed evaluation methodology, including evaluation matrix, approaches, data collection tools, and work plan (not more than 10 pages)

▪ Proposed budget proposal in USD (exclusive of costs related to field travel)

▪ A sample of the most recent similar evaluation report (written by the applicant)

▪ Updated curriculum vitae

For more details on the scope of the assignment, please refer to the following email vnazaria@iom.int.

Mandatory Subject title for applicants: “IDF TB Project Evaluation Consultant”

Other

Any offer made to the candidate in relation to this special vacancy notice is subject to funding confirmation.

Appointment will be subject to certification that the candidate is medically fit for appointment, security clearances and a valid COVID certification.

In accordance with project requirements, the eligible candidates should be only outside Republic of Moldova. A prerequisite for taking up the position is legal residency in the country of the duty station and work permit, as applicable.

Closing date for applications is 12 April 2024.

Only shortlisted candidates will be contacted.

Posting period:

From 26.03.2024  to 12.04.2024