Terms of reference for External Evaluation

TITLE

Caritas Moldova 2010023 CARING 

Total Value

EUR 784 924 (Caritas Österreich) + 103 860 (Caritas Moldova contribution)

Project Duration:

January 1^st 2021 – December 31^st 2023 

BACKGROUND

The project CARING - access to knowledge, care, education, safety and support for children and youth through care & support centers in Moldova – is implemented for 3 (three) years within 4 (four) centers in Tiraspol, Rashcov, Petropavlovca and Zirnesti. The challenge of achieving quality education, care and support remains a reality in all regions taking part in the project and such inequities and gaps are still present on family and school level including exclusion, violence, discrimination, lack of children’s participation, low parental engagement, and poor infrastructure.

The project influences the situation of more than 310 vulnerable children and youth, and approximately 100 parents, who both are the main target groups of the project, as well the capacity of 50 staff members.

The partners influencing proper project implementation are: Caritas Austria – donor, Roman Catholic Parish “The Holy Trinity”, Tiraspol city; Roman Catholic Parish “Saint Caetan”, Rashcov village; Roman Catholic Parish “The birth of the Blessed Virgin Mary ”, Grigorauca commune, Petropavlovca village; and Secondary School from Zirnesti commune.

The project aims to reduce the vulnerability to poverty of children and their families on both banks of Dniester River by increasing opportunities for learning and development in line with SDGs 2030 and key national policies in Child Protection.

To ensure a decent living and respect for the rights of the child, Caritas Moldova is focused on ensuring measures for the protection and support of children and youth in difficulty, by:

-           providing services for children and youth through Care & Support Centers, offering quality non-formal education programs based on individual needs and increasing their learning and social abilities.

-           promoting positive family environments and family values focused on respecting the principles of equal rights and nonviolent behavior and strengthening parenting skills in raising and educating children.

-           strengthening the capacities of the local staff/ stakeholders in the field of child rights protection, by sharing and expanding knowledge and improving the quality assistance provided to the child and his/her family.

OBJECTIVES

The final evaluation will focus on the following key objectives: 

• Assess the relevance, appropriateness, effectiveness’, accountability and impact/sustainability of the program.
• Assess the effectiveness of the efforts in mobilizing community support, fundraising, partnership forging, networking, etc.
• Identify strong and weak points of each center for children and recommend ways of strengths’ leverage.
• Assess the sustainability potential for each center/project.
• Assess the level of standards implementation within the centers for children (according to the Minimum Quality Standards of childcare facilities developed in 2020).
• Identify lessons learned, best practices (including stakeholders matrix) and recommendations to support future program design/development.
• Assess the impact.

Evaluation process will be looked at the following criteria and guiding questions: 

• Relevance

The extent to which the aid activity is suited to the priorities and policies of the target group, recipient, and donor. In evaluating the relevance of the project, the following questions should be considered:

• To what extent are the objectives of the program still valid?
• Are the activities and outputs of the program consistent with the overall goal and the attainment of its objectives?
• Are the activities and outputs of the program consistent with the intended impacts and effects?
• Was the program design based on an impartial assessment of needs? Are needs assessments disaggregated by age, sex and disability? Do they include people’s needs, vulnerabilities, and capacities?
• Did the services provided by Caritas Moldova meet the needs of the target groups? Were the people most in need identified, selected, and supported by the program?
• Which services/activities were the most appropriate and why? Which were least appropriate and why? Were activities aligned with the target groups’ needs and priorities?

• Effectiveness

A measure of the extent to which an aid activity attains its objectives. Consider the following questions:

• To what extent were the objectives achieved / are likely to be achieved?
• What were the major factors influencing the achievement or non-achievement of the objectives?
• Was the internal organizational and managerial structure of the project effective?
• Were there appropriate systems in place to monitor activities, outputs and outcomes of the program? Did monitoring outcomes inform program adjustments/revisions?
• Did the project activities lead towards the achievement of the expected results/indicators as set in the Logframe?

• Efficiency

Efficiency measures the outputs -- qualitative and quantitative -- in relation to the inputs. Consider the following questions:

• Were activities cost-efficient?
• Were objectives achieved on time?
• Was the project implemented in the most efficient way compared to alternatives?

• Impact

The positive and negative changes produced by the intervention, directly or indirectly, intended or unintended. This involves the main impacts and effects resulting from the activity on the local social, economic, environmental and other development indicators. The examination should be concerned with both intended and unintended results and must also include the positive and negative impact of external factors, such as changes in terms of trade and financial conditions. Consider the following questions:

• What has happened as a result of the project?
• What real difference has the activity made to the beneficiaries?
• How many people have been affected?
• Are the final users of services and products satisfied with the process and results?
• Has the program strengthened local capacities?
• What are the intended and unintended, positive and negative effects of the project?
• What, if any, aspects of the project will have a longer-term impact?

• Sustainability

Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable. Consider the following questions:

• To what extent did the benefits of the project continue after donor funding ceased?
• What were the major factors which influenced the achievement or non-achievement of sustainability of the project?
• Which of the efforts in mobilizing community support, fundraising, partnership forging, networking, etc. were appropriate and successful, and should be continued and developed?
• To what extent could each local partner ensure project continuation?
• What is the level of standards implementation within each center for children (according to the Minimum Quality Standards of childcare facilities developed in 2020).

• Accountability
• To what extent have the target groups been involved in the design or implementation of the project?
• Were appropriate systems of downwards accountability (participation, information sharing and feedback/complaints), put in place and used by project participants?
• Were project participants aware of the feedback/complaint’s mechanism?
• Were project participants and communities aware of selection criteria?
• Were project participants and communities aware of the services available for them?
• To what extent has accountability towards the main donor been assured?

• Coordination
• How effective was Caritas Moldova in coordinating internally?
• How effective was Caritas Moldova in coordinating with external stakeholders such as other agencies, organizations, the local and national government?
• How effective were local partners (the centers) in coordinating with external stakeholders such as other agencies, organizations, the local and national government?
• What aspects of coordination could be improved in the future and how?

Evaluation process and approach:

• Review project documents including the log frame, annual targets and indicators, and existing monitoring tools;
• Review the context behind the intervention(s) including – when pertinent – revision of legislation, cultural and social contexts;
• Assess relevance and effectiveness of the project’s strategy, methods and approaches for the achievement of the project objectives;
• Assess performance of the project in terms of effectiveness, efficiency, and timeliness of producing the expected outputs;
• Assess qualification of specialized staff;
• Assess project’s management arrangements: identify advantages, bottlenecks and lessons learnt with regard to the management arrangements;

The main methods to be used:

• Review of project documentation: This could include, but is not restricted to annual and quarterly reports, the original project proposal document, all relevant project data stored in organization system, training materials etc. This information will be provided to the evaluator(s) by Caritas Moldova and local partners. Any other relevant statistics and secondary sources should also be reviewed by the evaluator(s).
• Focus Groups Discussions;
• Key Informant Interviews Open-ended (semi-structured) interview;
• Visual reporting (pictures);

Stakeholders to be consulted: the consultant should plan to include the following groups of stakeholders in the evaluation:

• Direct project beneficiaries (possibly disaggregated by age and if appropriate by gender as described in the project documents).
• Project implementing staff.
• Other relevant staff of evaluated organization.
• Wider stakeholders – for example community leaders, local authorities representatives etc.
• Donor reprezenytatives.

METHODOLOGY

The final evaluation will rely on two main evaluation stages 1. design phase 2. field phase

Design phase 

• The evaluator(s) will undertake a desk review of programme documentation including planning documents, project proposals, situation reports and quarterly reports
• The evaluator(s) will review other monitoring and reporting documents from secondary sources.
• If appropriate, the evaluator(s) will obtain feedback from the CI Humanitarian Department on the most relevant sites to visit.

Field phase

• After the design phase, the evaluator(s) will conduct fieldwork to collect and analyses data in order to answer the evaluation questions
• Data collection methods should be inclusive and utilize a range of methods, including focus group discussions and key informant interviews with key project stakeholders. The use of surveys and other remote data collection tools should also be explored by the evaluators to maximize data collection.
• The evaluator(s) should ensure a systematic triangulation of data sources and data collection methods and tools and seek to validate data through regular exchanges with project staff where appropriate.

EXPECTED OUTPUTS

The evaluator(s) should produce the following key deliverables:

• Draft Evaluation Report in English language to be submitted to Caritas Moldova latest August 18^th 2023

 Final Evaluation Report will include:

Executive Summary

Background

Introduction

Context

Description of Methodology

Main findings

Conclusions inclusive of best practices and lessons learned

Recommendations.

USE OF THE EVALUATION RESULTS

The intended audience for the evaluation is Caritas Moldova key staff, including senior management, Caritas Austria, who has supported the project, and other national and international partners who contributed to the project. Evaluation findings will be shared with project participants as appropriate.

REQUIRED COMPETENCIES

The competencies required from the External Evaluator are:

• Advanced degree in social sciences, political sciences, economics, development or related fields.
• Experience in leading evaluations, especially in the field of social services, children protection and education.
• Ability to use participatory approaches to evaluation.
• Experience of operational management of education/development programs
• Good knowledge of the local context
• Good analytical skills
• Excellent writing skills in English Language
• Understanding of the Catholic Church and Caritas Moldova structure and mission.

CONSULTANT’S PROPOSAL 

Proposals should include:

- Proposed evaluation methodology;

- Description of the relevant experience of the bidder, including previous experience in evaluating similar projects;

- CV(s) of the team (the consultant can be a physical or a juridical person and can form a team of 1 or 2 experts with relevant experience);

- References and recommendations;

- Description of deliverables and timeline(according to the file from TIME FRIME);

- A financial proposal including the cost implication for the evaluators if any and CVs of evaluator (s).

The financial offer will be presented in MDL and will include all taxes, including the contributions of the
employer for paying salaries/fees for providing services in the Republic of Moldova.

TIME FRAME

Add as many lines as needed to prepare Time Frame plan Annex_1.  

Interested external evaluators are requested to send a “Letter of Interest” (offer) no later than July 19, 2023  to Caritas Moldova

EXTENDED DEDLINE: no later than July 28, 2023  

Office: 30/1 Gheorghe Asachi Street, Chisinau, MD-2004, Moldova

Offer should be sent to by e-mail at the address: procurement@caritas.md ,  tel.+37378800597

Please ensure that you receive a confirmation message that your files have been received. In the absence of this, Caritas Moldova Charitable Foundation is not responsible for electronic files sent but not received. 

Caritas Moldova reserves the right to cancel and/or suspend the procurement procedure, in whole or in part, and to reject all tenders at any time prior to the award of the contract, including in cases where it is unable to provide financial cover or in cases where tenders do not meet the requirements set out in this documentation, without thereby creating any obligations or liability towards tenderers.