PH International is seeking a consultant to conduct the final external evaluation for the Legal Socialization in Moldovan Schools Program

TERMS OF REFERENCE  

FINAL EXTERNAL EVALUATION

Legal Socialization in Moldovan Schools Program (MLSP)

Program title: Legal Socialization in Moldovan Schools Program

Duration: August 2019 – January 2024

Location: Republic of Moldova

Partners: The Ministry of Education and Research and General Police Inspectorate

Donor: The U.S. Department of State, Bureau of International Narcotics and Law Enforcement Affairs (INL)

BACKGROUND INFORMATION

In August 2019, Project Harmony Inc. (PH International), a U.S. based non-profit organization with an office in the Republic of Moldova was awarded a grant from the U.S. Department of State, Bureau of International Narcotics and Law Enforcement Affairs (INL) to support legal socialization and juvenile crime prevention in the Republic of Moldova.  The grant enabled the implementation of a two-year pilot initiative that expanded into a multiyear “Legal Socialization in Moldovan Schools Program” (MLSP).

The MLSP aims to strengthen the rule of law, support Moldovan communities in preventing juvenile delinquency, and develop a positive relationship between juveniles and law enforcement authorities by combining a legal education curriculum and afterschool activities, community-based initiatives to raise awareness about civic rights and responsibilities, deter crime by and against young people, and protect the rights of youth.

PH International uses a multi-dimensional approach that has gained the support and commitment of stakeholders from the highest ministerial levels, extending to local communities to prevent delinquencies by and against young people. The MLSP activities are based on the holistic and comprehensive approach of engaging police officers, together with teachers, in the legal education of young people in the classroom and through community-based activities.

The design of the MLSP aims to improve the legal understanding, knowledge, and skills of students in support of a society based on the rule of law. Students from 8^th grade (14-15 years old) and 11^th grade (17-18 years old) benefit from a wide range of activities that contribute to their development and the improvement of their communities. The activities include: Education for Legal Socialization optional subject (curriculum activity); STEM and Forensic clubs (extracurricular activity); school and community projects with police (extracurricular activity) in order to increase knowledge about the basic rights of citizens and encourage positive interactions with law enforcement institutions; Summer School “Police Explorer,” etc.

An innovative approach of the Legal Socialization (LS) program brings teaching teams made up of teachers and police officers together to work in schools to reach and impact youth. Youth benefit from close, one-on-one interactions with police officers in a non-confrontational, positive learning environment, and work closely with police and teachers on afterschool/community projects that benefit their schools and communities. Students now see law enforcement officers as mentors, rather than simply enforcers.

In the first two years (2019 – 2021 period) of implementation, MLSP piloted the Legal Socialization (LS) curriculum in 22 pilot schools from six regions of Moldova at the gymnasium level (8^th graders), and then expanded the pilot to 15 schools at the lyceum level (11^th graders).

In the following years, the LS curriculum implementation at the 8^th and 11^th grade levels was advanced as follows:

• In Year 3 (2021 – 2022 school year), 30 new schools were selected to implement the LS curriculum in 8^th grade classes, with  10 new schools selected for 11^th grade classes in 10 regions;
• In Year 4 (2022 – 2023 school year), 52 new schools (78 classes) were selected to implement the LS curriculum. In total, the LS curriculum was taught in 113 schools and 158 classes, targeting approximately 3,800 students in 20 regions;
• In Year 5 (2023 – 2024 school year), 30 new schools were selected to implement the LS curriculum for 8^th and 11^th

A multiplier effect of the program is the inclusion of enrichment activities that emphasize expanding and promoting legal socialization knowledge to peers, family, and the broader community. These enrichment activities include community projects to enhance public safety developed and implemented with police and community stakeholder involvement that build on community policing concepts; extra-curricular forensics clubs led by science teachers and police to engage students in using STEM subjects in criminal investigation and to help students to better understand the career possibilities open to them; and a Police Academy summer camp that provides skills and encouragement for students interested in a potential career in law enforcement. These activities aid in the prevention of juvenile crime and crime against juveniles as well as cultivate an interest in law enforcement as a career among young Moldovans. 

Program strategic outcomes are as follows:

1. Youth gain a greater understanding and respect for the rule of law, legal issues, and law enforcement authorities and better appreciate their role as citizens, thus reducing the likelihood of involvement in criminal or delinquent behavior.

2. Youth and public confidence in law enforcement is increased as skills and knowledge about the rule of law are applied in community settings in partnership with law enforcement authorities.

3. Partnerships among ministries and institutions working with youth on the rule of law issues are reinforced, and new relationships among these bodies, youth, and communities are established.

Key stakeholders for the MLSP are:

• Ministry of Education and Research (MER) and Educational Departments from MLSP regions;
• General Police Inspectorate and National Inspectorate for Public Safety;
• Local Police Inspectorates from MLSP regions;
• Selected primary and secondary schools from MLSP regions;
• NGOs and community organizations from MLSP regions.

Final beneficiaries:

• Students participating in the curriculum, enrichment, and community-based activities;
• School administration;
• Teachers and police officers teaching the LS curriculum activities and leading the extracurricular activities;
• Parents, who are indirect beneficiaries;
• Local communities, who are indirect beneficiaries.

The evaluation will focus on Year 3 and Year 4 of the MLS Program (2021 – 2023), however, it should take into account the conclusions from the previous intermediary External Evaluation for the period August 20, 2019 – August 20, 2021.

 Year 3 program components and activities:

Legal Socialization Curriculum development and delivery:

Expansion of LS curriculum to new schools and regions for 8^th and 11^th graders and finalization of the LS curriculum materials at the 11^th grade level; training of new teaching teams made up of teachers and police officers delivering the LS curriculum in schools.

 Extracurricular Activities with Police:

Support for School & Community Projects with Police:

Organization of action planning workshops for community engagement in solving community and school safety issues for police officers and school representatives from MLSP schools; provision of supplies, materials, and equipment in support of 20 school and community projects developed by students, teachers, and police to combat bullying, deter violence and improve safety; implementation of 3 community projects by selected NGOs that involve police, schools, and young people and extend the outreach of Legal Socialization activities and community policing at the community level.

STEM and Forensics Clubs:

Selection of new schools for participation in the STEM & Forensic Clubs; training of new teachers selected to lead the STEM & Forensic Clubs; update of curricular materials; provision of schools with upgraded forensic investigation toolkits; organization of the “Young Sherlock Holmes” competition for students; mentorship activities for the teachers, etc.

Police Academy Summer Camp:

Organize a “Police Explorer” summer camp to give LS students the opportunity to expand their interests and explore career opportunities in law enforcement.

 Year 4 and final year program components and activities:

The goal for Year 4 and final year is to expand the program impact in 20 regions and build on the sustainability of the program through stakeholder engagement at the municipal, regional, and national levels and build on the program achievements in Years 1, 2, and 3.

Development of the LS training program for the Police Academy:

Establish the Training Program for the teaching teams of the LS discipline at the Police Academy “Ștefan cel Mare” ASM/ Police Training Center (CIPAL); provide the ASM/CIPAL-led Training Program with equipment, manuals, and materials for teaching teams and schools; provide assistance in piloting the first training course for LS teaching teams organized by ASM/CIPAL.

Ensuring the sustainability and extending the impact of the Legal Socialization Program at regional and national levels:

Conduct reflection workshops on legal socialization topics with LS students; develop materials for the inclusion of LS subjects in other school curricula/ humanitarian disciplines; improve LS curriculum materials, supplementing with new materials; develop inter-institutional tools and practices for monitoring legal socialization activities at the regional level; develop Ambassadors of legal education made up of active LS teaching teams to promote legal literacy; develop LS - Team of the Year competition for teachers and police officers teaching the LS curriculum; develop and pilot the national LS competition for students; organize a study visit to US for key stakeholders to explore good practices of Legal Socialization and prevention of juvenile delinquency; organize a final conference to consolidate lessons learned and share the next steps.

Extracurricular Project Activities with Police:

Provide capacity building and community grants on a competitive basis to 10 NGOs for implementing projects that involve police, schools, and students and extend the outreach of Legal Socialization activities and community policing at the community level.

STEM & Forensic Extension and Sustainability:

Extension of STEM and Forensics Club to new schools; organization of capacity building and mentoring activities for the new teachers leading the new schools; organization of the national competition “Young Sherlock Holmes” for students; development of a teaching guide and other curricular materials for approving  STEM & Forensics as an optional discipline in the national curriculum, etc.

Police Youth Academy Summer Camps:

Organize “Police Explorer” summer camp to allow LS students to expand their interests and explore career opportunities in law enforcement.

OBJECTIVE OF THE EXTERNAL PROGRAM EVALUATION

Overall Objective

The evaluation will independently assess the overall MLSP implementation and achieved results.

The evaluation shall assess the activities, outputs, and outcomes of the MLSP initiative in the MLSP target regions following the Organization for Economic Co-operation and Development’s Development Assistance Committee (OECD DAC) criteria for evaluation: relevance, effectiveness, impact, and sustainability. The evaluation should also uphold the U.S. Department of State’s evaluation standards: usefulness, methodological rigor (evidence-based), and independence and integrity.

More specifically, the evaluation criteria need to objectively measure and reflect on the following dimensions of MLSP:

• Relevance of the MLSP strategy for preventing juvenile delinquency to attain the program objectives. (Were the right things done to achieve the desired outcomes? Are the activities, outputs, and outcomes of the program consistent with the program goal?)
• Satisfaction of the key stakeholders and the ultimate beneficiaries: police officers, police leadership, school principals, teachers, students, police officers, and parents. (Analysis of the implementation process and quality of technical support provided by the MLSP.)

• Effectiveness in achieving the planned results. (Were the right things done well? Have the intended results been achieved? What were the expectations of the key stakeholders and the beneficiaries?  Have the expectations been met?  What were the major factors influencing the achievement or non-achievement of the results?)

• Impact regarding the positive and negative changes produced by the program, directly or indirectly, intended or unintended. To what extent is it possible to measure the impact and end effect in terms of behavioral change among participating juveniles? that participated in the program activities  (What are the results of the program? Is there increased knowledge and understanding of legal issues and greater respect for the rule of law and law enforcement? Are there cases that indicate positive behavioral change? What were the major factors that influenced changes?)

• Evaluate what factors are likely to hinder or support the sustainability of MLSP outcomes beyond the program implementation period in the targeted regions. (What can be done, who could do it, and how would it ensure MLSP’s sustainability?)

• Identify possible next steps and recommendations for future opportunities. (What else could be done? what anticipated outcome(s) has the MLSP produced?)

 EVALUATION METHODOLOGY:

The final external evaluation will be conducted in several phases:

 Phase 1. Preparatory:

(i) Desk review of MLSP documents/reports (Notice of Funding Opportunity; technical application and follow-up proposals and work plans; logic model; monitoring and evaluation plan; quarterly reports; Intermediary Evaluation Report; Memoranda of Understanding between PH International and key stakeholders; curriculum materials; monitoring reports on the LS implementation and other extracurricular activities; U.S. Department of State Integrated Country Strategy for Moldova,  etc.)

(ii) Development of a detailed evaluation methodology and work plan;

(iii) Development of the evaluation tools;

(iv) Working sessions with PH Moldova/key stakeholders to validate and approve the methodology and evaluation tools.

Phase 2. Data collection (online and offline):

• (i) Key Informant Interviews (KII) with MLSP stakeholders in education law enforcement, including community policing officers, teachers, school principals, students, parents, NGO representatives, etc.
• (ii) Focus Group Discussions (FDG): with students, teachers, police officers, partner NGOs, etc.
• (iii) Questionnaires
• (iv) Other tools as identified by the external evaluator.

Phase 3. Report development and reporting which will include: (i) data analysis and interpretation of the obtained results; (ii) report preparation and discussion of the results of the external project evaluation with the PH team; (iii) presentation of the findings for validation with program partners in a working meeting; and (iv) presentation of the final evaluation report at the Final Conference expected to take place in the last week of November/first week of December 2023.

The evaluation team will use primary data sources (data collected through the KII, FGD, questionnaires, etc.) and secondary data sources (project data, statistical and administrative data from different sources, reports, etc.), to cross-check data and follow triangulation procedures.

ORGANIZATIONAL & MANAGEMENT STRUCTURE:

PH International has overall program management and accountability to INL, including implementation plans, financial and program reporting, monitoring and evaluation. PH has established a Curriculum Development Team (CDT) composed of nominated representatives from the Ministry of Education and Research and the General Police Inspectorate. The CDT also serves as the Program Advisory Council (PAC) for the MLSP and principal liaison to their institutions and other key stakeholders.

The PH International Country/MLSP Program Director, Program Coordinator, Program Assistant, and Financial Manager comprise the PH Moldova staff who provide program, administrative, and financial support for the implementation of MLSP.

DELIVERABLES:

• Evaluation methodology and workplan approved by the MLSP team;
• Final report: The consultant will develop an evaluation report (maximum of 50 pages, excluding the annexes).  The evaluation report will include, but not be limited, to the following sections: an executive summary, introduction, evaluation approach, evaluation findings, conclusions, recommendations, and annexes. The final evaluation report and all the annexes (e.g., evaluation tools) are required to be written in English. The consultant will also present an executive summary of the report in Romanian.
• Presentation: The consultant will make a short presentation of the final evaluation findings at the MLSP Final Conference.

 DURATION AND TIMEFRAME:

A tentative timeline for each evaluation phase is presented below:

TERMS AND CONDITIONS:

 PH International will contract the selected consultant (individual or legal entity) for an agreed-upon all-inclusive lump sum (consultant fee). PH will also cover the travel and accommodation costs for field visits, according to PH and US government allowances.

Details on payment conditions shall be defined in a contract signed between the selected consultant and the PH International Office in Moldova.

The evaluator should have the following professional background:

• University degree or professional experience in Social Sciences, Humanities, Law, or similar.
• Knowledge and understanding in one or more areas of community policing, juvenile delinquency approaches, human rights, education, etc.;
• Experience, knowledge, and expertise in conducting evaluations and surveys involving juveniles;
• Experience in managing and/or evaluating donor-funded programs and US government-funded programs in particular;
• Experience in qualitative and quantitative research methods;
• Excellent analytical and report-writing skills;
• Fluency in English and Romanian is mandatory. Knowledge of Russian would be an asset.

Data protection

In carrying out the evaluation of this project, the evaluator must respect the following confidentiality rule:

• All data relating to this project are confidential.

HOW TO APPLY:

Interested candidates should send their offers to the following addresses: maria.stratan@ph-int.org and ph-md@ph-int.org by October 2, 2023, with the subject line “MLSP final external evaluation”.

Candidates should apply by presenting the following documents:

1. CV(s) of the individual/for each team member indicating all past experience with similar assignments, as well as the contact details (email and telephone number) of the candidate(s) and professional references;
2. A Technical Proposal outlining the proposed evaluation methodology;
3. A Financial Proposal that indicates the all-inclusive fixed total contract price, supported by a breakdown of costs, taking into account the evaluation tools to be used. The Financial Proposal must be submitted in US dollars. Please provide the net amount. Please be aware that MLSP is categorized as zero-rated VAT (VAT 0%) by Government Decision.
4. Copies of documents proving the expertise (links) or a sample of an evaluation report conducted previously, preferably in English.

Incomplete offers and offers received after the deadline will not be considered.

The deadline for submitting the applications is October 2, 2023.

For requests for clarifications, please contact us by email or by phone:

Maria Stratan, Program Coordinator, mob: 069924194