Jesuit Refugee Service SEE / Renovabis

Terms of Reference for Evaluation of the project "Enhancing Hospitality in the Western Balkans" MOE 23815A

1. Background

The project, titled "Enhancing Hospitality in the Western Balkans" is a humanitarian initiative led by three of JRS South-East Europe (JRS-SEE) offices: JRS Bosnia and Herzegovina, JRS Kosovo, and JRS Serbia. The project aims to provide support and assistance to migrants and refugees in the Western Balkans, with a particular focus on vulnerable groups such as single women, children, youth, and families. Additionally, it seeks to promote integration, education, and advocacy efforts in the region. Context:

The Western Balkans have been a significant transit route for migrants and refugees, particularly since the onset of the European migrant crisis in 2015. Bosnia and Herzegovina, Kosovo, and Serbia have remained key countries on this migration route. The region has witnessed the arrival of thousands of refugees and forced migrants fleeing conflict, persecution, and economic hardship in the Middle East, Asia,and Africa.

JRS SEE, with its established presence in the Western Balkans, has been actively engaged in providing support, assistance, and advocacy for the rights of refugees and migrants. The project builds upon JRS SEE's experience, expertise, and network of volunteers and collaborators in the region.

The project's specific objectives are multi-faceted, addressing various aspects of the refugee and migrant experience:

• Legal framework and Integration: The project focuses on understanding the legal framework for international protection in Croatia, an EU member state, and draws parallels with Bosnia and Herzegovina, Kosovo, and Serbia, which are in different stages of EU integration. It emphasizes the importance of learning the local language for economic and social integration.
• Access to Education and Skills: The project aims to enhance access to education, strengthen IT skills, and facilitate employment and vocational training for refugees and migrants, ultimately promoting their integration into the labor market.
• Social Mentoring and Healthcare: It supports access to external services and healthcare, recognizing the importance of social mentoring in helping refugees navigate their new environment.
• Youth Inclusion: The project emphasizes the inclusion of young people and socialization with the assistance of young peer volunteers from the local population, fostering intercultural exchange.
• Media and Advocacy: It addresses the role of media in shaping public perceptions and attitudes toward refugees and migration, and it includes advocacy activities in Western Balkan countries.

Renovabis has been a consistent supporter of our projects since 2011, with the current project commencing on November 1, 2022. The evaluation was initiated at the donor'srequest, with a primary focus on the first year of the current project, MOE 23815A, spanning from November 1, 2022, to October 30, 2023. Additionally, it will encompass an examination of the two preceding projects, MOE

23815 and the Interim project MOE 23815B, to emphasize the evolution of activities and learning and provide a broader medium-term perspective. Primary purpose of the evaluation is to assess the relevance, coherence, effectiveness, efficiency, sustainability, and impact of the project "Enhancing Hospitality in the Western Balkans" MOE 23815A. This evaluation will provide valuable insights into the project's achievements and areas for improvement, contributing to informeddecision-making and the continued support of refugees and asylum seekers in the Western Balkans. It will also help inform future initiatives and strategies for addressing the needs and challenges faced by refugees and migrants the region.

1. Objectives and goals of the Evaluation:

1.1. Assess effectiveness and impact:

Evaluate the performance of JRS offices in Bosnia and Herzegovina, Kosovo, and Serbia in executing the "Enhancing Hospitality in the Western Balkans" project. Measure the project's effectiveness in achieving its goals and assess its impact on the integration and well-being of migrants and refugees, particularly vulnerable groups.

1.2. Identify strengths, weaknesses, and relevance:

Identify strengths and weaknesses in project design, implementation, and management. Assess the project's continued relevance amid changing migration dynamics and evolving refugee and migrant needs in the Western Balkans.

1.3. Inform futuredecision-making:

Provide evidence-based insights to inform future decision-making related to refugee and migrant support efforts in the Western Balkans. Guide strategic priorities and actions of JRS SEE three offices.

1.4. Enhance accountability:

Ensure accountability by evaluating project outcomes and resource utilization, ensuring responsible stewardship of resources, and fulfilling commitments to stakeholders and beneficiaries.

1.5. Facilitate learning and sustainability:

Serve as a learning tool for JRS SEE three offices, promoting a deeper understanding of successful strategies and areas for improvement in humanitarian efforts. Offer recommendations to enhance project sustainability.

1.6. Support informed decision-making:

Provide findings and recommendations that assist in resource allocation, project management, and decision-making at various levels, ensuring well-informed judgments.

2. Scope of Work:

The boundaries of the evaluation for the project "Enhancing Hospitality in the Western Balkans" are defined as follows:

2.1. Inclusions(Aspects to Be Covered):

2.1.1. Relevance:

In this section, the evaluation should assess the project's sustained relevance considering changing migration patterns in the Western Balkans. This entails determining if the project's goals correspond to the needs and priorities of migrants and refugees right now. The following questions should be answered by the evaluators:

• Has the project remained aligned with the evolving needs and priorities of migrants and refugees in the Western Balkans?
• To what extent have the project's goals corresponded to the current needs and priorities of the target beneficiaries?
• How effectively did the project's advocacy and awareness-raising activities contribute to a more positive and accepting perception of refugees in the Western Balkans?
• What impact did media coverage, workshops, and other initiatives have on changing attitudes and practices in the region?

2.1.2. Coherence:

The evaluation will focus on the effectiveness of advocacy and awareness-raising activities in promoting a more positive and accepting perception of refugees in the Western Balkans. It will assess the impact of media coverage, workshops, and other initiatives. Did the project's advocacy and awareness-raising activities align with broader regional or national strategies and initiatives related to refugee and migrant support?

• Were there opportunities for collaboration and synergy with other actors in the advocacy and awareness-raising space?
• Did the project's sustainability strategies align with national or regional efforts to support refugees and migrants?
• Were there opportunities for collaboration with other actors to enhance the sustainability of project outcomes?

2.1.3. Effectiveness:

Determining the project's success in attaining its stated objectives will be the main focus. This involves assessing how much the initiative has enhanced the integration and general well-being of immigrants and refugees in the Western Balkans. The following questions should be answered by the evaluators:

• To what extent has the project achieved its stated objectives in enhancing the integration and well- being of immigrants and refugees in the Western Balkans?
• What tangible outputs and outcomes can be attributed to the project, including changes in policies, practices, and attitudes?

2.1.4. Efficiency:

The following questions should be answered by the evaluators:

• Were project resources allocated efficiently, and were processes streamlined to achieve objectives cost-effectively?
• Were project timelines and budgets adhered to during implementation?

2.1.5. Sustainability:

The evaluation will look at various aspects of a project's long-term viability. It will investigate whether the project has laid the groundwork for sustainable improvements in the lives of beneficiaries. The following questions should be answered by the evaluators:

• Has the project laid the groundwork for sustainable improvements in the lives of beneficiaries beyond the project's duration?
• Are there mechanisms place to ensure that the positive changes brought about by the project will continue in the long term?

2.1.6. Impact:

All aspects of the project impact produced by the project (intended or unmittened, positive or negative) will be observed.

• To what extent have the basic needs of beneficiaries been met and improved?
• What changes have occurred in the lives of the target group beneficiaries as a result of the project?
• Did the project achieve some unintended effects (positive or negative)?
  1. Exclusions (Aspects Not Included):

Policy Analysis:

The evaluation won't go into detail about any lobbying initiatives or analyses of international migration or protection policies. It won't perform in-depth legal assessments, though it might briefly mention the system of international protection.

Political Context:

While acknowledging the importance of the political context in the Western Balkans, the evaluation will not engage in a deep political analysis of the region. It will focus primarily on the project's humanitarian aspects, and whether JRS’s efforts are align with national strategies, complementary to the activities of other actors in the sector/country, and whether there are synergies with other initiatives.

Previous Projects:

The evaluation will not extensively cover the historical projects of the JRS SEE three offices, except where relevant for context. The primary focus is on the specific activities and outcomes of the "Enhancing Hospitality in the Western Balkans" project.

Specific Areasof Focus:

The evaluation will seek to answer specific questions and address areas of focus, including but not limited to:

Economic and social integration (effectiveness/impact):

• To what extent have project activities improved the economic and social integration of migrants and refugees, with a specific focus on language learning, education, employment, and vocational training?

Well-being and social inclusion (impact):

• What impact has the project had on the well-being and social inclusion of vulnerable groups, particularly single women, children, and youth?

Access to services and healthcare (effectiveness):

• How effective has the project been in facilitating access to external services and healthcare for beneficiaries?

Role of youngpeer volunteers (efficiency and social inclusion):

• What role have young peer volunteers played in the socialization and inclusion of migrants and refugees, and to what extent have they been effective?

Influence on media narratives and public attitudes (coherence and effectiveness):

• To what extent has the project influenced media narratives and public attitudes regarding refugees and migration in the Western Balkans?
• What lessons can be learned from advocacy activities undertaken as part of the project in the countries of the Western Balkans?

Challenges and recommendations (efficiency and effectiveness):

• Are there specific challenges or obstacles encountered during project implementation?
• What recommendations can be made for improvement based on the challenges identified?

Methodology:

The evaluation of the "Enhancing Hospitality in the Western Balkans" project will employ a mixed- methods approach, combining both quantitative and qualitative research methods to gather comprehensive data and insights. The methods and approaches to be used are as follows:

• Document review: All project-related papers, such as project proposals, reports, and associated publications, will be thoroughly examined. This will provide the project a historical and contextual perspective.
• Surveys and questionnaires: The project's beneficiaries, including migrants, refugees, and project participants from the community, will be given structured surveys and questionnaires. These tools will be created to gather quantitative information on the effects of project-related activities.
• Key informant interviews: Detailed interviews with important project participants, such as project employees, partners, officials from the government, and representatives from civil society organizations, will be undertaken. These interviews will offer qualitative insights into the difficulties encountered throughout project implementation and its results.
• Focus group discussions: Focus group discussions will be organized with project beneficiaries, particularly vulnerable groups such as single women, children, and youth. These discussions will explore their experiences and perceptions regarding project interventions.
• Participant observation: Field visits will be conducted to observe project activities firsthand. This approach will enable the evaluator to assess the quality and effectiveness of activities in real-time.
• Data analysis: Quantitative data collected through surveys and questionnaires will be analysed using statistical software to identify trends, patterns, and correlations.
• Comparative analysis: The evaluation will compare project outcomes with baseline data and, where applicable, with similar projects or initiatives in the region to identify best practices and areas for improvement.
• Media analysis: In order to assess how the project is affecting media narratives and public perceptions, media content on migration and refugees in the Western Balkans will be examined. Content analysis of news articles, broadcasts, and social media will be required for this.

Reporting and deliverables:

The evaluation process will begin in November 2023, and conclude in March 2024, and the final report is expected to be ready in March 2024. All documents are to be delivered via email. A kick-off workshop and a debriefing workshop are foreseen at the beginning and at the end of the process, after sending the final report.

Expected Deliverables and Timeline:

Kick-off workshop

• Format: an online meeting via a platform supporting audio and video sharing.
• Introduction of the evaluation team with JRS SEE and Renovab is representatives.
• Approximately by end-October 2023
  

Inception report:

• Format: PDF
• This report will outline the evaluation's detailed work plan, methodologies, data collection instruments, and a refined timeline. It will also include any adjustments made to the initial ToR based on the project's specific requirements.
• Approximately by end-October 2023

Interim progress update:

• Format: PDF
• The interim progress update will provide a snapshot of the evaluation's progress up to that point. It will include information on data collection status, challenges encountered, and any necessary adjustments to the timeline or methods.
• Approximately by mid-January 2024

Draft evaluation report:

• Format: PDF
• The draft report will present the evaluation's findings, conclusions, and preliminary recommendations. It will include an analysis of the data collected, aligned with the evaluation criteria and indicators. The draft will be shared with key stakeholders for feedback.
• Approximately by February 2024

Stakeholder presentation:

• Format: In-person or virtual presentation (as per stakeholder preference).
• The lead evaluator will present the key finding sand preliminary recommendations to project staff, JRS national offices, and other relevant stakeholders. The presentation will allow for discussion and feedback.
• Approximately by late February 2024

Final evaluation report:

• Format: Comprehensive PDF document
• The final report will incorporate feedback received during the stakeholder presentation and any additional data analysis. It will include a clear executive summary, detailed findings, conclusions, recommendations, and annexes with supporting data.
• Approximately by March 2024

Expected language:

• The evaluation report should be available in English language.

Submission of all raw data:

• Format: Raw data files (e.g. Excel spreadsheets, interview transcripts)
• All raw data collected during the evaluation, including interview transcripts, survey responses, and any other primary data, will be submitted in their original format.
• Approximately by March 2024

Post-evaluation consultation:

• Format: Virtual meeting
• The lead evaluator will engage in a virtual meeting with key stakeholders to discuss the final report, clarify any questions or concerns, and provide guidance on the implementation of the recommendations.
• Approximately by late March 2024

Ethical Considerations:

Informed consent:

Informed consent will be obtained from all individuals participating in surveys, interviews, and focus groups. Participants will be provided with clear information about the purpose of the data collection, how their information will be used, and their rights to withdraw from the process at any time. Ensure that they understand the purpose of the evaluation, the data collection methods, and their rights regarding participation. Consent should be voluntary and documented.

Confidentiality:

All data collected, including personal information, will be treated with strict confidentiality. Data will be anonymized and stored securely to prevent unauthorized access.

Findings and reports will present aggregated data to ensure that no individual can be identified. Any direct quotes or personal stories used in the report will be anonymized.

Data security:

Data security measures will be implemented to protect the integrity and confidentiality of collected data. This includes secure data storage, password protection, and encryption of sensitive information.

Cultural sensitivity:

The evaluation team will be culturally sensitive and respectful of the diverse backgrounds and beliefs of participants. Efforts will be made to avoid any behavior or language that could be perceived as disrespectful or offensive.

Minimize harm:

The evaluation will prioritize the well-being of participants and aim to minimize any potential harm. Questions and discussions will be conduction a non-intrusive and supportive manner.

If any participant expresses distress or appears to be emotionally affected by the evaluation process, appropriate support and referrals to relevant services will be provided.

Beneficence:

Strive to maximize the benefits of the evaluation for both the project and the participants. Ensure that the findings and recommendations contribute positively to the project's goals and the well-being of beneficiaries.

Transparency:

The evaluation process, including its objectives, methods, and potential impacts, will be transparently communicated to stakeholders and participants. This transparency fosters trust and understanding.

Feedback mechanisms:

A mechanism for participants to provide feedback or raise concerns about the evaluation process will be established. This ensures that any issues can be addressed promptly.

Respect for autonomy:

Participants will have the autonomy to decline participation or withdraw from the evaluation at any point with outfacing negative consequences.

Evaluation team qualifications:

A competent and diverse evaluation team is crucial for the success of the evaluation process. Each team member should posses specific qualifications and expertise relevant to their roles and responsibilities. Here is an outline of the qualifications and roles:

Lead evaluator:

• Advanced degree in a relevant field (e.g., social sciences, international development, evaluation).
• Extensive experience in project evaluation, particularly in the context of migration and refugee support programs in one of the countries in which this project is conducted.
• Strong analytical and research skills.
• Ability to manage the evaluation process, coordinate team members, and ensure the quality of deliverables.
• Excellent communication and report-writing skills.

Support evaluators:

• In-depth knowledge of refugee support programs, advocacy efforts, and the specific challenges faced by refugees, migrants, and asylum seekers in their local context.

Contact information and deadline:

1. For inquiries related to the Terms of Reference (ToR) and the overall management of the evaluation process for the project "Enhancing Hospitality in the Western Balkans," please feel free to reach out to JRS South-East Europe contact:

• Name: Stanko Perica
• Email: stanko.perica@jrs.net

Offers deadline:

• Deadline for submission of the offers is 20 October2023.

Total costs of evaluation

• 10,000.00 Euro

JRS South-East Europe Zagreb, 5 October, 2023