Scope of the Evaluation
The outcome evaluation seeks to:
- Review the UNDP Timor Leste Governance Programme with a view to understand its relevance and contribution to national priorities for stock taking and lesson learning, and recommending corrections that may be required for enhancing effectiveness of UNDP’s development assistance;
- Review the status of the outcome and the key factors that have affected (both positively and negatively, contributing and constraining) the outcome;
- Assess the extent to which UNDP outputs and implementation arrangements have been effective for building capacities of key institutions which implement government schemes and policies (the nature and extent of the contribution of key partners and the role and effectiveness of partnership strategies in the outcome);
- Review and assess the Programme’s partnership with the government bodies, civil society and private sector and international organizations in Programme;
- Review links/joint activities with other UNDP Programmes and UN Agencies and how these have contributed to the achievement of the outcome;
- Provide recommendations for future Country Programme regarding ways in which the UNDP resources can most strategically impact change in capacities of key institutions of the country so that the delivery mechanisms of the Government are better designed, suit their purpose, and that governance systems put inclusion at the centre of Government efforts;
- Through this evaluation UNDP Timor Leste seeks to understand and articulate the key contributions that the Governance programme has made in the programmes on democratic governance processes, a rigorous analysis of the areas of synergy between the various capacity development strategies adopted within the programme and with other practice areas of UNDP Timor Leste and recommendations to strengthen UNDP’s interventions in this critical area of engagement with the Government of Timor Leste.
The outcome evaluation will be conducted in 2018, and will thus, provide strategic direction and inputs to adequately prepare the United Nations Development Assistance Framework (UNDAF) and UNDP Country Programme after 2020. UNDP will incorporate the findings of the evaluation, experiences and lessons learned while preparing the new Country Programme Document. This evaluation is also expected to bring recommendations regarding partnership to inform a resource mobilization strategy and implementation strategies which could help to (re-)design modalities and management arrangements.
Outputs/Deliverables of the Evaluation:
The inception report which details the evaluators understanding of the evaluation and how the evaluation questions will be addressed. This is to ensure that the evaluator and the stakeholders have a shared understanding of the evaluation. The inception report will include the evaluation matrix summarizing the evaluation design, methodology, evaluation questions, key informants, data sources and collection analysis tools for each data source and the measure by which each question will be evaluated. The inception report is to be discussed with the UNDP Country Office.
Draft Governance Outcome Evaluation Report:
To be put forward during pre-validation workshop (30 -50 pages). The report will be reviewed by all stakeholders to ensure that the evaluation meet quality criteria. The suggested table of content of the report is as follows:
- Executive summary;
- Introduction (Background and approach/methodology);
- Development context (Presentation of issues in the country context. Factual);
- Development results (Presentation of findings based on the evaluation criteria and other cross-cutting issues. Analytical.);
- Bottlenecks for and drivers of success (Analytical);
- Conclusions and Recommendations;
- Final Governance Outcome Evaluation Report, integrating feedback voiced during pre-validation workshop 10 days after receiving the draft report.
The proposed evaluation approach is outlined below. Should there be changes to the approach by the evaluation team, they need to be in accordance to the international criteria and professional norms and standards adopted by the UN Evaluation Group, and previously approved by the UNDP. The Outcome Evaluation will be carried out in accordance with UNEG Evaluations Norms and Standards for Evaluation and OECD/DAG Principles.
The evaluation team will visit selected project sites to meet the local stakeholders and beneficiaries including Central, State and district government officials, civil society organizations including resource agencies, local authorities including elected women representatives, women leaders with potential to contest assembly elections, academics and subject experts, community members and volunteers, etc.
The evaluation shall assess the following for each outcome in the 2015-2019 programming cycle in this portfolio:
- Relevance: Are the project and programme outcomes relevant to UNDP’s mandate, to national priorities and to beneficiaries’ needs? (Relevance to UNDP’s country programme);
- Effectiveness: Have the intended impacts been achieved or are they expected to be achieved? Do different outcome definitions feed into each other and is there a synergy in between? Is the outcome achieved or has progress been made to achieve? Has UNDP made significant contributions in terms of strategic outputs?;
- Efficiency: To what extent do the outcomes derive from efficient use of resources? And to what extent UNDP has contributed to the outcomes versus that of its partners?;
- Degree of Change: What are the positive or negative, intended or unintended changes brought about by UNDP’s intervention in these outcomes?;
- Sustainability: Will benefits/activities continue after the programme cycle?.
All evaluation questions should include an assessment of the extent to which programme design, implementation and monitoring have taken the following cross cutting issues into consideration:
- Human rights;
- Gender Equality;
- Capacity development;
- Institutional strengthening;
- Innovation or added value to national development;
- South-South Cooperation.
The outcome evaluation will include the following key activities:
- Evaluation design and workplan (to be shared with UNDP Timor Leste before start of the evaluation);
- Desk review of existing documents;
- Briefing with UNDP Timor Leste;
- Field visits;
- Interviews with partners;
- Drafting of the evaluation report;
- Debriefing with UNDP Timor Leste;
- Finalization of the evaluation report (incorporating comments received on first draft).
Though the evaluation methodology to be used will be finalized in consultation with the UNDP Timor Leste Country Office, the following elements should be taken into account for the gathering and analysis of data:
- Pre-assessment of data availability
- Desk review of relevant documents including Country Programme Document (CPD), Country Programme Action Plan (CPAP), Project Documents/ Briefs, project evaluations (interim and final), reports of relevant flagship projects, etc.;
- Discussions with the Senior Management and programme staff of UNDP Timor Leste;
- Presentation of an inception report and discussion of the content with UNDP management and partners;
- Interviews: with key partners and stakeholders both at central and field levels. Drawing on specific check listed questions supporting the DAC criteria;
- Focus group discussions: within UNDP and external parties both at central and field levels. Gaining consensus on key issues;
- Participation and providing guidance to an Outcome Board Meeting of the Democratic Governance outcome;
- Field visits to select key projects, if necessary;
- Regular consultation meetings with the UNDP staff, project staff and senior management as appropriate.
This evaluation exercise will be undertaken by an Evaluation Expert. The Evaluation Expert will be reporting to the Country Director of UNDP Timor Leste, with delegated authority to the Governance Programme Analyst. The Governance Programme Analyst will arrange the introductory meetings within UNDP and will establish the first contacts with the government partners and project staff. The expert will then set up his/her own meetings and conduct his/her own methodology upon approval of the methodology submitted in the inception report.
The draft and final reports will be submitted in English. The expert will work home/office-based with presence in UNDP premises in Dili as needed for the desk reviews and will get support (contacts and vehicle) from UNDP in arranging their travel arrangements for the site visits.
The Evaluation Expert shall plan all the resources he/she needs to complete the assignment within their financial proposal to the UNDP. The resources to be used by the expert shall be subject to UNDP approval.
The Evaluation Expert will have the overall responsibility for the conduction of the evaluation exercise as well as quality and timely submission of the final evaluation report to UNDP. S/he will specifically undertake the following tasks:
- Lead and manage the evaluation mission;
- Design the detailed evaluations scope, methodology and approach;
- Conduct the outcome evaluation in accordance with the proposed objective and scope of evaluation;
- Draft, communicate and finalize the evaluation report as per the comments from UNDP Timor Leste.
Required Skills and Experience
- Minimum Master’s degree in economics, public administration, regional development/planning or any other social sciences related to governance issues especially decentralization and local governance, capacity development and democracy;
- At least 10 years of experience in conducting evaluations, preferably of programmes focused on decentralized governance, accountability, capacity development and access to justice;
- Strong working knowledge of UNDP and its mandate, the civil society and working with government authorities;
- Extensive knowledge of results-based management evaluation, as well as participatory M&E methodologies and approaches;
- Experience in applying SMART (S Specific; M Measurable; A Achievable; R Relevant; T Time-bound) indicators and reconstructing or validating baseline scenarios;
- Minimum 10-15 years of professional experience in the area of development, including democratic governance, poverty reduction, regional development, gender equality and social policies;
- Strong reporting and communication skills;
- Excellent communication skills with various partners including donors.
Please be sure to indicate that you saw this position on Globaljobs.org