Consultancy: Team Member/Research Assistant for the Study of Influential Evaluations 2024, 6-months (87 days), Evaluation Office, NYHQ

UNICEF’s Evaluation Policy states that, “Evaluation at UNICEF unequivocally serves the organization’s mission and supports UNICEF in fulfilling its mandate.” The Evaluation Policy further states that, “to be useful, evaluations must contribute to organizational learning, accountability and to decision-making.” In line with these policy statements, evaluation in UNICEF is to be considered a vehicle towards contributing to tangible change for children. Evaluations in UNICEF are thus expected to be influential. UNICEF evaluations play a crucial role in informing decision-making processes for both internal and external stakeholders. The organization's evaluations provide evidence-based information and analysis of UNICEF programs, policies, and interventions. They also assess their relevance, effectiveness, efficiency, and sustainability, among other factors.

UNICEF works in some of the world’s toughest places, to reach the world’s most disadvantaged children. To save their lives. To defend their rights. To help them fulfill their potential. 

Across 190 countries and territories, we work for every child, everywhere, every day, to build a better world for everyone. 

And we never give up. 

For every child, evaluate

UNICEF’s Evaluation Policy states that, “Evaluation at UNICEF unequivocally serves the organization’s mission and supports UNICEF in fulfilling its mandate1.” The Evaluation Policy further states that, “to be useful, evaluations must contribute to organizational learning, accountability and to decision-making.” In line with these policy statements, evaluation in UNICEF is to be considered a vehicle towards contributing to tangible change for children. Evaluations in UNICEF are thus expected to be influential.

UNICEF evaluations play a crucial role in informing decision-making processes for both internal and external stakeholders. The organization's evaluations provide evidence-based information and analysis of UNICEF programs, policies, and interventions. They also assess their relevance, effectiveness, efficiency, and sustainability, among other factors. Therefore, it is important for evaluations to be influential in decision-making for several reasons:

1. Improving program and policy effectiveness: UNICEF evaluations provide valuable insights into the impact of its programs and policies. Decision-makers can also use evaluation findings to identify best practices, assess program effectiveness, and make informed decisions about how to allocate resources (both scaling up and scaling down). In addition, UNICEF collaborates with governments to generate evidence to improve policy design and effectiveness by means of evaluation.
2. Enhancing accountability and transparency: UNICEF evaluations provide a mechanism for accountability and transparency. By sharing evaluation results with stakeholders, UNICEF demonstrates its commitment to transparency and openness.
3. Promoting learning and knowledge sharing: UNICEF evaluations provide a place for learning and knowledge sharing. By communicating evaluation findings, UNICEF promotes learning and knowledge sharing among stakeholders, including UNICEF staff, donors and partners.
4. Strengthening partnerships and collaboration: UNICEF evaluations provide an opportunity to strengthen partnerships and collaboration. By involving stakeholders in evaluation processes, UNICEF can build trust and strengthen relationships with partners.

UNICEF Evaluation for Decision-Making

The purpose of evaluation is to ensure that UNICEF has timely, strategically focused, and objective information on the performance of its policies, programs, and initiatives to produce better results for children and their caretakers. It is to strengthen evidence-based decision-making and advocacy, transparency, coherence, and effectiveness. Furthermore, the purpose and use of evaluation at UNICEF is to determine the relevance, impact, effectiveness, efficiency and sustainability of the interventions and contributions of the organizations of the United Nations system.

UNICEF conducts evaluations at five institutional levels reflecting the organizational accountability framework: (i)local or project, (ii) country program of cooperation, (iii) regional, (iv) global strategic and (v) institutional performance2. These include the following 3 typologies of evaluations:

1. Thematic Evaluations: These evaluations are developed in consultation with the Global Evaluation Committee, as well as divisions and offices at the global, regional, and country levels. Independent Corporate Evaluations gather and synthesize evaluation evidence on priority topics related to promoting and protecting children's rights, with a specific focus on the five Goal Areas outlined in the UNICEF Strategic Plan.
2. Impact Evaluations: Impact evaluation (IE) aims to empirically establish attribution by quantifying causal links between the intervention (projects, programs, policies, networks or capacity development) and outcomes of interest, typically at the targeted beneficiary level. It does so by establishing a counterfactual scenario.
3. Humanitarian Evaluations: UNICEF's humanitarian evaluations are systematic assessments conducted in emergency and crisis settings to understand the effectiveness, efficiency, and relevance of UNICEF's humanitarian response efforts. These evaluations focus on evaluating the impact and outcomes of UNICEF's interventions in meeting the urgent needs of children and their families during times of crisis, such as natural disasters, conflicts, epidemics, or displacement.
4. Country-led Evaluations: A UNICEF country-led evaluation is an evaluation conducted at the country level, where the national government leads the assessment with support and guidance from UNICEF. This type of evaluation focuses on programs, policies, or interventions related to children's welfare and rights, which UNICEF supports or is involved in. The key features of a country-led evaluation include strong national ownership, alignment with national priorities and systems, and capacity-building for local evaluation professionals.

UNICEF evaluations remain crucial in informing decision-making processes, improving program, institutional and policy effectiveness, enhancing accountability and transparency, promoting learning and knowledge sharing, and strengthening partnerships and collaboration. Thus, it remains important for UNICEF evaluations to be influential in decision-making, particularly in helping to improve the conditions and situations of the most vulnerable children and, more broadly, to ensure that UNICEF programs and policies are evidence-based, effective, efficient, and sustainable. To what extent are UNICEF evaluations catalyzing the changes expected from them? This question remains to be more comprehensively answered. Although there have been a few assessments aimed, among other things, at determining the extent to which evaluations were being used, these have been mainly focused on the degree of implementation of evaluation recommendations.

To make further progress towards answering the extent to which UNICEF evaluations are being influential, the Evaluation Office undertook 2 separate studies, in 2018 and 2020 to examine the extent to which a sample of UNICEF evaluations had been influential.

The studies concluded that, most commonly, the influence was instrumental, leading to the improved design and implementation of policies and programs. In other cases, the evaluations prompted the re-conceptualization of long-standing programs, leading to the comprehensive reform of policies. What distinguished the influential evaluations was the scale and scope of change achieved. Among the most salient outcomes was the increase in government funding and ownership, showing that evaluations can contribute to sustainability - an area that has been flagged for improvement in UNICEF.

The study reaffirmed the need for additional evidence to determine the influence of evaluations and to ensure that evaluations are purposefully planned and designed to have an impact on decision making.

Types and Criteria for Influence

The relevant literature has highlighted several factors that should be considered to determine whether an UNICEF evaluation has been influential in decision-making. One key type of influence is strategic influence, where evaluations shape the overall direction of a program or policy by providing evidence-based insights and recommendations. This often involves assessing the relevance and sustainability of initiatives. Another important type is operational influence, where evaluations lead to improvements in the day-to-day management and implementation of programs. This may involve identifying best practices or areas for improvement. A third type is learning influence, aimed at enhancing knowledge and understanding among stakeholders, which can lead to better decision-making and policy formulation. These evaluations must be credible, unbiased, and transparent to effectively inform policy and program decisions, ensuring that interventions are aligned with the needs and priorities of the targeted populations. Table 1 below describe these influential aspects in more detail:

Table 1: Determinants of Influence3

We also include an adapted version of assessing influence by applying the OECD evaluation criteria. These criteria ensure a thorough and balanced assessment, capturing both the immediate outcomes and the longer-term effects of an evaluation, making them an important for understanding and enhancing an evaluation’s influence. 

1 UNICEF (2023). Revised Evaluation Policy for UNICEF. 4 Aug 2023. Second regular session 2023. N2322917.pdf (un.org)

2 At UNICEF, institutional performance or effectiveness evaluations primary assess the overall performance and efficiency of the organization. These evaluations focus on measure the extent to which an UNICEF’s goals and objectives are being achieved, its ability to utilize resources effectively, and its capacity to adapt to changing circumstances. The evaluations may include aspects such as governance, leadership, strategic planning, financial management, and administrative systems. The goal is to enhance UNICEF’s function and optimize its operations.

3 Applied from UNICEF 2020 Study of Influential Evaluations: McPherson and Kumar.

How can you make a difference? 

The purpose of this study is to determine the most influential evaluations that were conducted by UNICEF between the period of 2020-2022; and to analyze what are the enabling or constraining factors leading to influence. Influence will be analyzed in terms of evaluations’ contributions to the policy environment, policy change (public policies as well as organizational policies), strategic shifts, improved programming and performance, organizational learning, joint work, among others. Findings, conclusions, and recommendations will be used to improve the quality and intent of evaluations in UNICEF; and to develop guidelines to help achieve greater levels of influence through evaluations. The final expected outcome of this study will be to provide valuable insights into the factors that contribute to the influence of evaluations. Moreover, it will help UNICEF to improve its evaluation practice and to produce evaluations that are more likely to be influential.

The expected results of the study intend to:

1. Identify positive (enablers) and negative (bottlenecks and barriers) that are deemed relevant to influence an intervention.
2. Identify best practices and lessons learned for an evaluation to be considered "influential".
3. Internally verify how program design and implementation improved by means of evidence generation and use.
4. Identify which major enablers were the most important areas of influence such as resource mobilization, partnerships, advocacy, coalition building, evaluative culture, strengthening abilities for evaluation, leadership, management, governance, knowledge management, etc.

To qualify as an advocate for every child you will have… 

• An advanced university degree (Master’s or higher) in development, social sciences or related disciplines.
• A minimum of 5 years of relevant professional experience in evaluation or in evaluative studies.
• Experience in using Natural Language Processing for text mining.
• Experience in analyzing qualitative data.
• Strong analytical and writing skills.
• Ability to perform wide stakeholder consultations.
• Developing country work experience and/or familiarity with emergency is considered an asset. 
• Fluency in English is required. Working knowledge of Spanish and French is desirable. Knowledge of another official UN language (Arabic, Chinese, or Russian) or a local language is an asset. 

How to apply:

• CV and cover letter.
• The daily rate and availability within duration of this assignment.
• A test may be administered to shortlisted candidates.
• The shortlisted candidates may be asked to submit short examples (of completed analyses or published papers, in line with the required qualifications described above.

For every Child, you demonstrate… 

UNICEF's values of Care, Respect, Integrity, Trust, Accountability, and Sustainability (CRITAS). 

To view our competency framework, please visit  here. 

UNICEF is here to serve the world’s most disadvantaged children and our global workforce must reflect the diversity of those children. The UNICEF family is committed to include everyone, irrespective of their race/ethnicity, age, disability, gender identity, sexual orientation, religion, nationality, socio-economic background, or any other personal characteristic.

UNICEF offers reasonable accommodation for consultants/individual contractors with disabilities. This may include, for example, accessible software, travel assistance for missions or personal attendants. We encourage you to disclose your disability during your application in case you need reasonable accommodation during the selection process and afterwards in your assignment. 

UNICEF has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UNICEF, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination. UNICEF also adheres to strict child safeguarding principles. All selected candidates will be expected to adhere to these standards and principles and will therefore undergo rigorous reference and background checks. Background checks will include the verification of academic credential(s) and employment history. Selected candidates may be required to provide additional information to conduct a background check. 

Remarks:  

Individuals engaged under a consultancy or individual contract will not be considered “staff members” under the Staff Regulations and Rules of the United Nations and UNICEF’s policies and procedures, and will not be entitled to benefits provided therein (such as leave entitlements and medical insurance coverage). Their conditions of service will be governed by their contract and the General Conditions of Contracts for the Services of Consultants and Individual Contractors. Consultants and individual contractors are responsible for determining their tax liabilities and for the payment of any taxes and/or duties, in accordance with local or other applicable laws. 

The selected candidate is solely responsible to ensure that the visa (applicable) and health insurance required to perform the duties of the contract are valid for the entire period of the contract. Selected candidates are subject to confirmation of fully-vaccinated status against SARS-CoV-2 (Covid-19) with a World Health Organization (WHO)-endorsed vaccine, which must be met prior to taking up the assignment. It does not apply to consultants who will work remotely and are not expected to work on or visit UNICEF premises, programme delivery locations or directly interact with communities UNICEF works with, nor to travel to perform functions for UNICEF for the duration of their consultancy contracts.