Call for External Evaluation of the Project Social Justice
1. Description of the intervention
Context: Roma, Ashkali and Egyptian people are trapped in a vicious cycle of exclusion. They do not enjoy their rights and lack opportunity, without adequate health, social protection and care. They are not empowered to be part of societal decision making. All of these are sustained by discrimination and antigypsyism which leads to disengagement, demotivation and disinterest - again reinforcing exclusion.
Overall Goal of the Project: The Project Consortium’s overall goal is a truly multi-ethnic and multi-cultural society in Kosovo, where Roma, Ashkali and Egyptian people are treated as equals by the state and society, with access to the same opportunities as everyone else, and where Roma, Ashkali and Egyptian people know their rights and their responsibilities and are empowered to exercise those rights. A Kosovo where Roma, Ashkali and Egyptian are in decent employment in both public and private sector, where they have access and make use of decent quality public infrastructure, where services are available and accessible locally and where Roma, Ashkali and Egyptian women are less oppressed and under-represented in education and employment and in other spheres of life and where they are more empowered.
Theory of Change: The Consortium recognises that three main pathways to change are necessary to make progress towards this long-term vision, which can be summarized as follows:
IF
the policies and institutional framework for social and economic inclusion of Roma, Ashkali and Egyptian people in Kosovo are strengthened; and
antigypsyism is addressed institutionally, among service providers and in the public sector and, at the same time, Roma, Ashkali and Egyptian communities are empowered to recognize practices, deal with their fears, overcome their passivity and strengthen their trust in institutions; and
integrated actions for social inclusion are implemented and the barriers to participation for Roma, Ashkali and Egyptian communities - for children and women in particular - are addressed and people become empowered to be agents of their own change
THEN
the integrated actions towards social inclusion will become more and more effective and the quality of life of Roma, Ashkali and Egyptian people in Kosovo will improve, step by step. Roma, Ashkali and Egyptian people will be treated as equal and in a position to exercise the same opportunities and rights as everybody else. As a consequence, the gap between Roma, Ashkali and Egyptian and non-R/A/E in educational outcomes, access to employment and decent living conditions will diminish over time and the overall inclusion of Roma, Ashkali and Egyptians will improve and increase.
The following outcomes were defined for the project:
Outcome 1: Mainstreaming social inclusion policies and practices has progressed and improvements have been made in procedures in key policy areas that impact on the lives on Roma Ashkali and Egyptian people – namely (i) educational learning centres (ii) the scholarship programme (iii) child protection systems (iv) legal framework for redressing antigypsyism (v) employment quotas for ethnic minorities, (vi) affordable housing solutions (vii) cadastral plans for settlements.
Outcome 2: There is evidence of some shift in public opinion and there is growing recognition of antigypsyism in media and public sector that it is s a root cause of exclusion of RAE people that needs a legal framework and a process and system to redress its negative impact.
Outcome 3: The combined actions applied by the Consortium including, community based childcare centres, tailored labour market interventions and empowerment of Roma, Ashkali and Egyptian women have improved the overall inclusion of Roma, Ashkali and Egyptian people in Kosovo
Project Set-Up: The project components are allocated between VoRAE and Tdh based on the operational strengths of each organisation and supported by HEKS.
The Directors of each organisation play a critical role in the process, but so too does every member of the implementation teams . It is their inputs that ensure the programme’s influence and outcomes are as far reaching as needed – extending from grass-roots level in communities up to the level of ministries, agencies and private sector businesses in Kosovo and beyond in the wider European context.
The implementation teams as well as the Consortium leadership team meet on monthly basis, discuss challenges and opportunities to project implementation and take decisions. In addition, the implementation teams established functional partnerships and exchange mechanisms with other R/A/E inclusion projects in Kosovo and with others operating in similar thematic areas, to avoid duplication and create synergies and maximize the overall project results.
For strategic steering, a project steering strategic committee is formed, consisting of representatives of the project partners (VoRAE, HEKS/EPER, Tdh and Swiss Agency for Development and Cooperation (SDC) and the key partners at the national level such as Office for Good Governance, Prime Minister's Legal Office, Ministry of Labour and Social Welfare, Ministry of Education, Science and Technology.
2. Purpose and scope
Purpose: The purpose of the evaluation is to provide the consortium with external views and insights about the effectiveness,efficiency and sustainability of key project interventions and advise on possibilities on how to best build on, sharpen and prioritize them, to work as effectively as possible towards the long term goal and outcomes of the project in the coming phase.
Building on the internal mid-term review of the project conducted in 2021 and given the broad scope of the project, the consortium has decided to focus the external evaluation on interventions where additional information is needed to sharpen, prioritize and adapt interventions for the next phase. This concerns first the new interventions in the field of antigypsyism, where the assessment of first experiences is crucial, second the interventions addressing labour market inclusion of Roma, Ashkali and Egyptians, because here the consortium would like to sharpen the approach to increase effectiveness and to complement efforts of other actors in the field, and third the interventions on child protection, because the consortium would like to document and make accessible the learnings for future political dialogue and advocacy work in other policy areas. As each of these areas requires specific qualifications by the evaluator, the assignement has been divided along these lines (please see below 4. and 10. for details).
More specifically, the evaluation seeks to achieve the following objectives:
Assess the effectiveness of the interventions to contribute to the three project outcomes, as specified in the project document, with specific focus on the new interventions in the field of antigypsyism.
Assess the effectiveness, efficiency and sustainability of interventions in the field of labour market inclusion as a basis for the Consortium for deciding on future focus/priorization of interventions.
Assess the relevance, effectiveness and sustainability of interventions on child protection in terms of closing protection gaps and working towards social inclusion and equality of Roma, Ashkali and Egyptians to provide learnings and perspectives for other aspects of social protectionProvide suggestions for further improving the cooperation in the consortium and the M&E system of the project.
Target audience: project consortium, SDC, major back-donors (Medicor Foundation, Bachmann Foundation); in addition, information can be shared with other civil society actors and relevant line ministries of the Kosovo Government.
The purpose of this evaluation is for both accountability and learning purposes that will in turn serve for strategic project steering. However, given the potential application of the findings for the next project phase, learning is prioritized in the evaluation processes. Therefore, the evaluation is mainly formative and the results will be used to develop the next project phase. The results of this evaluation will be complemented by external consultancies in the areas of early childhood development and social housing/settlement transformation, which will complete the information needed for rethinking the sharpening and priorization of the project interventions in the next phase.
Evidence Base: The evaluation shall take into account existing evidence on social inclusion of Roma and other vulnerable groups in Kosovo, which is available through the yearly governmental reports on the implementation of the National Strategy for the Inclusion of Roma and Ashkali, studies conducted by other donors, the MICS from 2019 (which was used for some baseline data by the project) as well as the Antigypsyism Baseline Study conducted in 2021 and the Consortium’s internal mid-term evaluation exercise from November 2021. This can be completed by other sources suggested by the consultant.
3. Criteria and questions
The evaluation questions were developed by the project team of the Consortium (please refer to the table on page 5-7) and address the DAC criteria of effectiveness, efficiency and sustainability/next steps.
4. Methodology and process
The overall approach is expected to be based mainly on a mix of qualitative methods for data collection and analysis but making strategic use of internally and externally available quantitative data for the analysis and interpretation. The evaluation can build on existing project monitoring data.
The evaluation team is expected to use methods such as: analysis of documents and of existing statistical data and budgets, structured interviews, semi-structured interviews face-to face or by phone, focus group discussions (FDGs), etc.
Selection strategies for the different project components are to be defined as part of the inception phase.
The evaluation will be divided in three components with thematical focus requiring specific qualifications. The project consortium encourages applicants to put together a team or to jointly apply with other consultants so that all qualifications can be covered. In case two or more consultants apply together, coordination needs to be clarified.
The process should follow the following steps:
Contract and Kick-off meeting: Signing of contract discussion of the assignment with the Consortium partners. First documents, including available data, are provided to the evaluation team.
Inception phase: forms part of reaching final agreement on methodological tools and procedures, including selection of interview partners / sampling strategy. Includes desk study and key informant interviews (by phone/skype) with key project staff, and possibly additional external stakeholders to be agreed on in the kick-off meeting.
A template for an inception report, including an evaluation matrix, is attached as annex 2. The inception report will be reviewed by all consortium partners.
The evaluators will organize one-day workshop where key findings and first conclusions should be discussed with key staff of the consortium and SDC.
Report writing: Submission of draft final report, inclusion of comments by consortium. The subsequent drafting process may go through additional feedback rounds until the final report is approved. The conclusions have to address the evaluation questions in the TOR, taking into account also unexpected finding. It is expected that the evaluation/review team will present concrete recommendations which are addressed to the specific stakeholders.
Data triangulation and quality control are very important and need to be discussed in the inception report. The field visits will only take place upon official approval of the inception report by the contractor.
The following table shows the evaluation questions and specifies to which project outcome (and in some cases outputs) each question refers. The methodology for addressing each cluster of questions should be developed by the evaluation team as part of the inception phase.
5. Deliverables
The evaluator(s) is expected to produce the following deliverables:
An inception report containing the evaluation design and design methodology (including attached research framework/evaluation matrix)).
Data collection tools
A final evaluation report responding to the evaluation objectives and questions; its structure will be outlined and agreed upon as part of the inception report. The final report shall have maximum 25 pages without annexes
Final PowerPoint presentation which summarizes the content of the evaluation report.
Full data analysis
Raw data (questionnaires, documentation of interviews, lists, data base)
6. Schedule and budget
The bidder is free to propose the number of days he/she needs for the evaluation; however, we wish to recommend that the total working days for the whole should not exceed 30 days in total, or 12 days for component 1 and 8-10 days each for components 2 and 3. If the different assignments are done by the same evaluator, it is suggested that respective field phases for the different assignments are conducted in parallel by different team members.
The consultant is expected to submit his / her daily rate, including VAT (if applicable), as part of the application. The consultant may apply together with a local/international partner/consultant. In this case, the daily rates of all team members should be indicated. Accommodation, travel and related costs will be reimbursed.
7. Management roles and responsibilities
The evaluation is mandated by the project Consortium – VoRAE, HEKS/EPER and TdH. The overall coordination of the contract is in the hands of VoRAE as a lead Consortium member. For VoRAE, the responsible person is Isak Skenderi, the Executive Director, for HEKS is Angela Elmiger, Program Manager for Kosovo and for Tdh Florina Duli Sefaj Tdh Country Director for Kosovo.
The consortium will assign a contact focal point to liaise with the selected consultant. This person will be responsible for:
Management of communications between the consortium and the consultant and between the consultant and partners-beneficiaries.
Organization and provision of the internal documentation required by the consultant.
Logistic arrangements regarding the organization of field work.
Approve the deliverables in consultation with consortium members and notify to the Consortium lead to process the installments associated to each of them.
Provide feedback of Consortium on every aspect of the evaluation, and they will provide the consultant with all required documents. Provide the consultant with any necessary logistical support (transport, lodging, etc.).
The consultant has responsibility for:
Preparing the overall evaluation design,
Propose a methodology and tools (in consultation with all stakeholders),
Suggest selection and sampling proceduresDevelop the data collection tools,
Plan the data collection, brief the local team,
Conduct the data collection, analyse all data, and write the reports (inception, final).
The consortium will:
Provide the evaluator with all necessary project documents and organise a project presentation/briefing with key project staff.
Collect comprehensive background documentation and inform partners and selected project counterparts
Provide a list of key stakeholders and their contact details
The program staff members will be responsible for liaising with partners, logistical backstopping and providing relevant documentation and feedback to the evaluation team
8. Follow up of the assessment
Upon finalization of the evaluation report, a management response will be developed by the Consortium. The management response will respond to the recommendations and outline a plan of action for implementing accepted recommendations from the report.
As part of the evaluation process, the consultant is expected to facilitate a feedback and reflection workshop for the Consortium and SDC. Results and findings from the evaluation will be used in the development of the next project phase and will be shared with the project design team for consolidation.
9. List of documents
Project proposal
Log-Frame
Overall Budget
Annual Planning and Annual Reports
National Roma and Ashkali Integration Strategy
Reports on Implementation of the Strategy of Roma and Ashkali Inclusion
World Bank/UNDP Roma Survey Report
Key documents of the Institutionalization Process in the field of child protection
Agendas and reports of important events
MICS Study 2019 (which was used for some baseline data by the project)
Antigypsyism Baseline Study conducted in 2021
10. Assessment team / qualifications
The bidder can either apply for one or two assignments of this evaluation or propose an evaluation team qualified for evaluating the questions of all three assignments according to the, of skills and experience mix to adequately deliver on these ToRs, and the specific set of evaluation question indicated in the list of evaluation list questions above.
In case separate bidders are selected for individual assignments, the evaluators are expected to coordinate how they can cooperate for the reflection workshop.
The evaluation team should be able to comply with the following list of required qualifications, competencies and experiences:
Skills and experiences in at least three project evaluations in relevant thematic topic (Roma inclusion with a focus on addressing antigypsyism, labor market inclusion, child protection)
Proven experience in the collection and analysis of both quantitative and qualitative data
Post-graduate degree in Social Sciences, International Development, Development Studies, Social Welfare, Social Policy, Social Work or related fields, Human Rights Studies, Economic Development Studies.
Demonstrated knowledge of the political, cultural, and economic situation in the Eastern Europe region with a specific focus on social inclusion for Roma programmes
Extensive knowledge of local government structures and programmes in Kosovo.
A deep understanding of social services system in Kosovo.
Proven expertise in the evaluation of projects.
Selected consultants must not have been involved in the design, implementation, or monitoring of this project.
11. How to Apply
Interested candidates should present:
A technical offer (max. 5 pages, plus annexes) comprising:
Proven understanding of the objective of the evaluation and the Terms of Reference (ToR);
Key questions of the evaluation
Brief description of the methodology and tools proposed to answer each of the key questions;
A chronogram showing details for the realization of each of the evaluation phases, based on the above table (steps to be adapted as necessary, submission of final report no later than May 15 2022).
Distribution of tasks if application as a team
A financial offer with a detailed cost breakdown as necessary.
An updated CV (of both team members, in case the proposal entails a pair submitting)
At least two evaluation reports in similar fields
Contacts of 3 references
Applicants should send the complete documentation in electronic form to i.skenderi@vorae.org (cc to Angela.Elmiger@heks.ch and florina.sefaj@tdh.ch) by 16th March 2022 COB.
https://kosovajob.com/vorae/call-for-external-evaluation-of-the-project-social-justice
Call for External Evaluation of the Project Social Justice
