External Evaluator for a Health & Nutrition programme – Central African Republic

Sub Levels


Purpose of the Evaluation

The purpose of the evaluation is to assess to what degree the project “Improved maternal and child health and nutrition levels in two sub-prefectures of Ombella M’Poko, Central African Republic” has been successful in achieving the intended results/outcomes and the contribution of the project in improving the living conditions of targeted beneficiaries. It is a fact that evaluations at Concern are primarily for learning and accountability. Thus, this evaluation will be carried out to establish good practices and lessons learned to develop stronger and more successful interventions in the future. This will be an external evaluation led and facilitated by a consultant hired by CONCERN CAR. The key findings of this evaluation will be shared, through clusters, with partners operating in the same sectors and dealing with the same context. As a final evaluation, DAC criteria (Coherence, relevance, Efficiency, Effectiveness, Impact and sustainability) will be used to measure the performance of the project through multiple lenses.

Description of the Context

The Central African Republic (CAR) is one of the poorest and most fragile countries, which has suffered for several years from poor governance and weak structural investments. It is ranked 188th out of 189 countries according to the United Nations Development Project (UNDP) Human Development Report (2020). Life expectancy at birth is 53.3 years, the lowest value recorded in the Human Development Index (HDI). On the other hand, according to the Humanitarian Response Plan (HRP) 2021, more than half of the population needs humanitarian assistance, including 1.9 million people who are in critical condition due to the deep and complex crisis that the country continues to experience. In 2018 the national SMART survey showed alarming rates of severe malnutrition among children under five. 10 out of 16 prefectures have SAM rates above 2%. In 2019, it was estimated that more than 110,000 children under five will need urgent nutritional support, including nearly 45,000 children with SAM, 65,000 with MAM and 65,000 with diarrhoea. In addition, the CAR has one of the highest maternal mortality rates in the world, 882 deaths per 100,000 live births (2015 est.), while the under-five mortality rate is 121 per 1,000 births. According to the CAR Humanitarian Needs Overview (HNO) 2021, the prevalence of Global Acute Malnutrition (GAM) – 5.8% in 2019 at the national level – is considered “precarious” and that of Severe Acute Malnutrition (SAM) – 1.3% in 2019 – is at the threshold according to the WHO classification. The area targeted by this project is part of the 10 prefectures with a prevalence of chronic malnutrition above 40%. The « Improved maternal and child health and nutrition levels in two sub-prefectures of Ombella M’Poko, Central African Republic » project matches that nutritional situation.

Regarding accessibility and security, numerous armed groups continue to exert their influence throughout the country, creating tensions and regular clashes for control of specific areas. The country has faced a protracted crisis since 2013. The presence of armed groups is a fundamental obstacle to the economic development of the Central African population, limiting their free movement and access to essential services. In addition, the level of insecurity coupled with the lack of transport infrastructure restricts the humanitarian response. In December 2020, the security crisis somewhat disrupted the security crisis. The main roads were closed, many residents had fled to settle in other villages, and most activities were suspended until May 2021. In addition, the coronavirus pandemic (COVID-19) further weakened the Central African population and exacerbated existing vulnerabilities. Today in 2023, although the security context is relatively calm, it remains fragile and volatile.

Description of the subject of the evaluation.

The project Improved maternal and child health and nutrition levels in two sub-prefectures of Ombella M’Poko, Central African Republic, is a 36 months project (September 2020 – August 2023) funded by UK Aid Match (Foreign and Commonwealth Development Office or FCDO). It was implemented in the Bossembele health district, particularly in 5 health facilities: 4 health facilities (Pama, Ngaramon, Bouyembé, Gbawi) and 1 hospital (Yaloké).

This project aimed to respond to the significant nutrition challenges in two sub-prefectures of the OMBELLA POKO prefecture (Bossembélé, Yaloké). Through the activities of this project, Concern provides support in the management of severe and moderate acute malnutrition (SAM) of children under five years old Concern’s nutritional response focuses on the prevention and treatment of acute malnutrition among the most vulnerable groups: such as children under five and pregnant and lactating mothers through the therapeutic and supplementary feeding program as well as communication for behaviour change through the promotion infant and young child feeding (IYCF). This project aimed to increase access to healthcare services and the use of qualified midwives for pregnant women, i) access to prevention, care and treatment of childhood illnesses in children under five and levels of breastfeeding; ii) reduce the levels of morbidity and mortality of children under five and pregnant women in the target areas. It targets 12,500 direct beneficiaries, including 10,080 children and 2,420 pregnant/breastfeeding women. Several activities have been realised under this project:

  • Health Assistance through the equipment of health facilities. Construction of 2 CMAM rooms in Bouyembe and Gbawi (Yaloke), Construction of 4 incinerators in Bouyembe, Gbawi, Ngaramon and Pama, Light rehabilitation of health facilities in Bouyembe, Ngaramon, including yet-to-be-finalised constructions: Construction of two maternities in Gbawi and Pama, Construction of two maternity yards in Gbawi and Pama.
  • Nutrition through training of health professionals, community-wide sensitisation and the development of referral pathways focusing on Maternal Infant and Young Child Nutrition. Nutrition activities were concentrated on Maternal Infant and Young Child Nutrition through a community-led approach. 50 Community Health Workers (Relais Communautaires, RECOs) and light mothers have been identified and trained to lead mass sensitisation and support groups for pregnant and lactating women (PLW) and their partners and to screen and referral of cases of malnutrition in children aged 6-59 months. Sensitisation will also focus on nutrition through culinary demonstrations and WASH to encourage community members to adopt good hygiene practices to save their lives. Support had been given to 4 FOSAs Bouyembe, Gbawi, Ngaramon, Pama, and the Secondary Hospital of Yaloke in drugs and Plumpy’Nut for the care of beneficiaries, etc.

Evaluation objectives and scope

This evaluation aims to assess all the project components, the level to which the project satisfied its targeted outcomes and establish the key learning points for future interventions. This will be based on DAC criteria and will be pursuing the following specific objectives:

  • Assess if the project has targeted extreme poor and vulnerable people;
  • Assess progress made towards the achievement of objectives, indicators and targets and the appropriateness of the targeting strategy used in this context;
  • To what extent have the activities interventions adhered to planned implementation and achieved intended goals, purposed and outcomes?
  • Support the organisation’s commitment to accountability to donors, government, public, and beneficiaries through assessing the good practices and gaps and adjusting the learnings in future projects;
  • Assess the progress made by participants regarding self-sustainability and resilience to future crises.
  • To which extent were participants, community members and other stakeholders associated with programme design and implementation
  • Measure the performance of the intervention regarding the seven DAC criteria: the coherence, relevance, efficiency, effectiveness, coverage, connectedness, and impact of the project according to a grading scale.
  • Identify lessons learned and provide practical and innovative project options for sustainably assisting returnees based on integration and/or relocation.
  • Provide recommendations on how the intervention could be improved in future.

The evaluation will focus on all engaged stakeholders (beneficiaries, governmental partners, implementation partners, and local authorities). This will be conducted in all intervention areas. Therefore, based on the security context during the evaluation period, the more accessible areas might be prioritised.

Evaluation Type

The evaluation will be a performance evaluation using a mixed-methods approach. This will be led and facilitated by an external consultant. It will focus on all project components and activities, how they have been implemented, the performance and the level of involvement. Learning and improvement will be the two keys focus of this evaluation.

Evaluation questions

Evaluation questions will be based on the DAC criteria of Coherence, Relevance, Effectiveness, Impact and Sustainability and Learning. The following questions will guide the consultant during the process:

Coherence

  • Was the invention carried out in coordination with others, and what was the extent to which the intervention added value while avoiding duplication of effort?
  • What were the synergies and interlinkages between the intervention being evaluated and other interventions carried out by CONCERN or other partners, as well as the consistency of the intervention with the relevant international norms and standards to which Concern adheres?

Relevance: Is the intervention doing the right thing?

  • Were the outcomes and associated project relevant, appropriate and strategic to national goals and Concern policies and guidelines?
  • Was there an appropriate contextual analysis carried out to inform project design, which was based on Concern’s Understanding of Extreme Poverty?
  • Were interventions appropriate and effective for the target group based on their needs?
  • Which target groups and individuals were reached by the interventions?
  • How practical was the targeting approach in achieving the project goal?

Efficiency

  • Were resources (both human and financial) used well? Could things have been done differently, and how?
  • Did the project’s activities reach the appropriate target groups and individuals within the target areas?
  • What were the major factors influencing the achievement or non-achievement of the objectives?
  • Did the monitoring system use capture the correct information at the appropriate times throughout the project?

Effectiveness: Is the intervention achieving its objectives?

  • To what extent do the project’s interventions appear to have achieved their intended outputs and outcomes?
  • Were the risks identified correctly in a Risk Management Plan and then mitigated?
  • What are the major factors influencing the achievement or non-achievement of the objectives?
  • How has management adapted the project design or implementation based on monitoring information and feedback from the target population?

Impact: What difference does the intervention make?

  • What changes—expected and unexpected, positive and negative—were experienced by the targeted beneficiaries and other stakeholders?
  • Has the project enhanced the capacity of the community institutions?
  • How have the project interventions impacted differently on men and women, boys and girls, people with disabilities (and other vulnerable groups as identified) in the project area?

Sustainability and learning: Will the benefits last, and what are key learnings from this project?

  • To what extent did the project take advantage of other USG and non-USG investments in the same target areas to facilitate linkages with complementary services, layering with earlier investments, and implementing an exit strategy?
  • Was the project able to end operations at the close of the award without causing significant disruptions in the targeted communities?
  • How might we do things better in the future? Which findings may be relevant for future programming or other similar initiatives elsewhere?
  • What were the key learning from this project (success factors, best practices, lessons learned)?
  • What has been Concern’s exit/transition strategy, and did it work? What are key recommendations in that regard?

In addition to the DAC criteria mentioned above, the evaluation should analyse in depth the following cross-cutting aspects by demonstrating to what extent they have been taken into account in the project and/or how the intervention logic, as well as the activities of the project, have integrated the following cross-cutting components:

  • Gender/Equality;
  • Sensitivity to conflicts;
  • Protection;
  • Partnerships, especially with the health ministry;
  • Environment and climate change

Methodology

The final evaluation will be a performance evaluation which will use a mixed-method approach based on the project’s logical framework and take into account the proposed evaluative questions in section 5. A detailed evaluation methodology will be presented by the consultant to be validated by CONCERN’s technical committee. This will include necessarily the following points:

  • Project documentation review. The consultant will have to review all existing documentation regarding the project for instance: contextual analysis reports, project documents, periodic reports, reports of previous evaluations and any other document that can facilitate his/her understanding and knowledge of the project and the mandate assigned to him. This also includes briefing with Concern staff.
  • Sample approach and size: The consultant will specify clearly in his/her methodology all the information related to the sampling, the number of individual interviews, key informants, and focus groups to be carried out. For the quantitative part, the consultant will propose a detailed sampling plan. The sample should be representative of the whole population of beneficiaries meeting the following characteristics e= 5% (error of margin); Z= 95% (confidence level). Also, a clear sampling plan for the qualitative study will be proposed by the consultant, including a range of different stakeholders from different geographic areas
  • Data collection and tools: For data collection, a final mixed-methods approach which consists of both quantitative and qualitative data collection, will be used. This will integrate a comparison of baseline and endline quantitative data as well as a qualitative study. The quantitative part will be based on the logical framework of the narrative document that the evaluator will use to measure the results and effect of this intervention on the direct and/or indirect beneficiaries and compare the outcomes from baseline to endline. The qualitative part will consist of conducting focus groups (FGDs) and semi-structured interviews with the beneficiaries, project team, community members, community leaders, local authorities, Health care staff, RECOs, and other stakeholders (local, national, governmental) to address specific key standards evaluation questions including questions about the following cross-cutting issues and themes: gender/equality; protection; partnerships; sensitivity to conflicts. The consultant will use CONCERN’s existing tools and other pertinent tools to gather the information that will help better interpret and triangulate the quantitative results. The detailed methodology will include these tools and a clear sampling plan.
  • Data analysis: The consultant will precise the types of analysis that he/she will make. For instance: statistical tests, correlation analysis, level of significance, mean comparison, etc. The tools he/she will use to analyse both quantitative and qualitative data.

Expected products/outputs

The consultant must provide several documents/outputs, including the following:

  • The technical proposition (maximum 20 pages). The latter will present the different stages of the evaluation process, from planning to submission and presentation of the final report. This will also include a transparent methodology, tools, and a detailed project plan against a timeline.
  • The financial proposition: The consultant will submit with the technical bid a detailed budget, including his/her daily fees, government tax, and all costs related to the evaluation.
  • The inception report (after documents review)
  • The final report of the evaluation (Paper version, Maximum 45 pages): This will include the results, conclusions, lessons learned and recommendations in terms of actions to be taken in the future in similar projects. The evaluation report must also contain an executive summary no more than three pages, and its long version must analyse the DAC criteria and the six cross-cutting aspects mentioned above and provide an assessment of performance against each of them using the following rating scale: 5 – Outstanding performance; 4: Very good overall performance with few shortcomings; 3 – Good overall performance but with some minor shortcomings; 2: Generally acceptable performance but with some major shortcomings; 1: Barely acceptable performance with many major shortcomings; 0: Completely unacceptable performance or insufficient data to make an assessment.
  • An electronic version of the final report will be shared internally with CONCERN and other partners operating in these sectors. This report must be submitted in French and in English.
  • A summary of the final report (maximum 3 pages): This will be used for communication and external publications on the CONCERN networks and/or other appropriate platforms;
  • A PowerPoint presentation (maximum 25 slides): The consultant will elaborate and share a Powerpoint presentation in which the methodology used, and the key findings of the study, the learnings and the key recommendations will be included;
  • The databases (raw and cleaned). At the end of the evaluation, the consultant will share with CONCERN team all the databases. These will be kept by CONCERN as evidence and used for other additional analyses if necessary.
  • Any other deliverable according to the deadlines set by the calendar (minutes of meetings, visit report, etc.)

Line of Communication

The consultant will report directly to the Area Coordinator and the Program Director, but will collaborate with the Project Manager and the MEAL Manager. Also, for any technical or specific questions, the consultant will be in contact with the technical advisors in Dublin. Any important decision relating to the evaluation will be taken jointly with the CONCERN technical committee, including the Program Director, Area Coordinator, Project Manager, and MEAL Manager Supported by technical advisors from Dublin.

Working conditions

The working conditions of the consultant are presented as follows:

  • The consultant will work in the CONCERN office in Bossembele office, with visits to the field (Bossembele, Yaloke) or to Bangui in accordance with the validated work plan;
  • The consultant must have their own laptop to carry out their work;
  • Payment of the consultant fees will be made in two parts of 50% each. The first upon signature of the contract and the second after validation of the final report.
  • Travel costs (international and domestic flights, visa, COVID tests), accommodation and field travel will be provided in accordance with CONCERN CAR policy;
  • The consultant must fulfil all formal CONCERN administrative requirements, such as signing the Project Participant Protection Policy (4Ps), code of conduct, child protection policy, …;
  • The consultant must make sure to have health and repatriation insurance in the rules and is responsible for its payment;
  • The consultant must comply with the CAR security policy and security procedures in the country. Any breach of this obligation will result in the immediate termination of the contract.

Composition, skills and experience of the consultant

  • At least a master’s degree in Statistics, social sciences, nutrition, biostatistics or related fields;
  • Five (5) to ten (10) years of experience and expertise in project development, Nutrition project evaluation (at least 5 years);
  • Proven expertise with final evaluations of projects using a mixed methodology (quantitative and qualitative);
  • Experience working in complex and rural areas, and under-developed countries; previous experience in CAR is desirable;
  • Proven expertise in manipulating statistical software for quantitative data management, such as SPSS, R, Stata, etc.
  • Strong analytical skills;
  • Good knowledge of Health and nutrition programming;
  • Fluency in both spoken and written French and English

Plan for Evaluation implementation

The consultant will propose a detailed calendar considering the following elements: document review, field visits, data collection and analysis, debriefing, and final submission of the evaluation report.

Week 1

  • Travel to Bangui
  • Brief meeting (Skype if need be) with Concern staff
  • Review of relevant documents
  • Develop the evaluation framework
  • Develop the data collection checklist
  • Prepare inception report with a detailed schedule

Week 2&3

  • Travel to Bossembele
  • Recruitment and training of enumerators
  • Data collection (HH Surveys, FGD, KII, … with gender, disability and age disaggregation where possible)

External evaluator, with facilitation by Concern

Week 4

  • Travel to Bangui
  • Data analysis, prepare and submit first preliminary results to Concern for feedback
  • Travel to his/her country

Week 5&6

  • Data analysis, prepare and submit first draft report to Concern for feedback
  • Produce the final report incorporating all the feedback from Concern staff
  • Report submission to Concern and final restitution

How to apply

Applicants are invited to submit their full proposition (technical and financial) to provide the services described above. They must also provide information relating to their abilities and experience, demonstrating that they are qualified to carry out this work. (CVs, professional references, copies and/or links of at least two final nutrition project evaluations already carried out, etc.).

Applicants have to send their applications to the following email addresses: Leo.Vuillermoz@concern.net ; Valerie.Zutphen@concern.net before 12 May 2023 at 23:59 mentioning the project’s title: « Improved maternal and child health and nutrition levels in Ombella M’Poko». Any late application will not be considered.

The application must include the following documents:

  1. Cover Letter,
  2. Curriculum Vitae, including professional references
  3. Detailed technical proposition
  4. Detailed financial proposition (budget)
  5. At least an example of a previous evaluation report conducted



Source link

Leave a Reply

Your email address will not be published. Required fields are marked *