AWRAD is grateful for the opportunity provided by the Search for Common Ground to undertake the independent evaluation of the Regional Health Cooperative Initiative. The team is particularly grateful to Rawan Assaly, the SFCG DM&E Coordinator, Wajdi Bkeirat, Health Project Manager at the SFCG, Sari Husseini, MECIDS Executive Officer at SFCG and Sharon Rosen, SFCG Co-Director for their time and reflections on the experience with the Regional Health Cooperative Initiative. The team is equally grateful to all regional MECIDS health representatives, trainers, laboratory technicians and public health officials and others who made themselves available to meet with the Evaluation team, and offered their views and suggestions.
List of Tables and Charts Tables
Category of respondents compared with type of tool used
Scores of training participants by nationality
Tabulations of changes in Israeli’s perceptions of understanding and attitudes from baseline to end line
Percentage of junior lab technicians showing increased positive attitudes to training counterparts at project end line
Change in positive attitudes from baseline to end line by gender of junior lab technician
Positive changes in understanding and attitudes from baseline to end line by nationality of TOTs
Percentage of senior lab technicians showing increased positive attitudes to training counterparts at project end line.
Percentage of senior lab technicians who had positive perceptions of CM and Technical training components
Change in positive attitudes of senior lab technicians by gender from baseline to project end line
Self-reported changes in ability to deal with conflict and disagreement from baseline to end line by nationality
Self-reported changes in ability to deal with conflict and disagreement from baseline to project end line by gender
Self-reported changes in CM skills between baseline and project end line by seniority of lab technician.
List of Acronyms
Arab World for Research and Development
Connecting Organizations for Regional Disease Surveillance
Middle East Consortium on Infectious Disease Surveillance
Palestinian Ministry of Health
Regional Cooperative Health Initiative
Search for Common Ground
Terms of Reference
Trainers of Trainers
United States Agency for International Development
EXECUTIVE SUMMARY Background The main goal of the Regional Cooperative Health Initiative (RCHI) is to build professional and personal understanding among senior public health officials and technicians in the Israeli, Jordanian and Palestinian public health sector. This goal was pursued by building institutional and individual cooperation among the Israeli, Jordanian and Palestinians public health sectors on biosafety and, at the same time, harmonizing biosafety levels in Israel, Jordan and Palestine. Project activities included training of trainers in biosafety techniques and conflict mitigation (CM) skills, training of lab technicians from the three participating countries to Biosafety Level 3 (BSL3), and a scenario simulation of a biosafety outbreak for senior public health officials of the three countries.
The overarching aim of this final evaluation is to capture the change brought about by the project among its target groups and beneficiaries in terms of increased professional capacities, CM skills and attitudes towards their regional counterparts. The specific objectives of the final evaluation include assessing progress towards results, assessing overall project performance against key criteria, and tabling lessons learned and recommendations for future projects of this type.
Methodology The frame of reference for the evaluation was based on three of the five basic Organizations for Economic Cooperation and Development criteria of relevance, effectiveness, and sustainability. The evaluation was undertaken in three phases (i) the inception phase, to plan and scope the evaluation and develop the evaluation tools, (ii) the data collection phase, which included documentation review, baseline and end line surveys, interviews and focus groups in Jerusalem and Cyprus, and (iii) the analysis and reporting phase, during which the team analyzed and synthesized all the collected data and prepared this evaluation report.
Analysis of Results and Key Findings
The vast majority of findings confirm the utility of the current project. Across all evaluation criteria (effectiveness, relevance and sustainability), the project exhibits positive achievements. The scores for indicators show improvement on most fronts, with the need for improvement in some eras in the future as indicated below.
In terms of Effectiveness,findings show thatfour of the project results have been completely achieved and one has been partially achieved. The overall project goal was well achieved.
The professional capacities of all health technicians (23 were trained as trainers [TOTs] at bio safety level 3 [BSL3] and 59 were trained by these trainers) trainings were significantly increased. On the basis of their scores in the final exam, all 82 received certification at BSL3 and four of the TOTs can now provide BSL3 training at international level. All senior public health officials involved in the simulation exercise felt that the experience of modeling a real emergency taught them essential communication and response protocols for managing common action. This result was fully achieved.
Trainers from the three countries trained mixed groups from the three countries. The final average scores on the exam of participants from the three countries were within the same range (82-85) indicating that biosafety trainings of health technicians in the three countries are at the same standard. This result was fully achieved.
Health officials were able to coordinate and cooperate together during a modeled emergency. The simulation exercise was highly rated by all participants and trainers. The exercise produced a common action plan for management of emergencies as an outcome. This result was fully achieved.
Cooperation was built on both the individual and institutional levels among all levels of health officials and technicians from the three countries. All MECIDS members from the three levels claimed increased levels of communication with each other, and almost all junior lab technicians and TOTs reported a desire to ‘stay in touch with other participants’ (91.3% for juniors, 90.4% for TOTs) and a willingness to ‘work on common issues’ (93% for juniors , 96.3% for TOTs). TOTs also reported increased traction for networking on the personal level (from 67% to 92% at end line). This result was fully achieved.
The CM skills of health technicians from the three countries overall decreased, although those of the Israelis increased. This result was only partially achieved.
In general, the project succeeded in creating effective networking opportunities between health officials and technicians. MECIDS has shown that these networking opportunities can be sustained at the highest levels of the public health system. Sustaining networking opportunities at the lower levels of health technicians will, to some extent, depend on the leadership of the senior public health officials. In the public health sector, practitioners tend to stay in their jobs for life, which helps to sustain the networking.
In terms of Relevance, the participants were in consensus that the project and its goals are relevant, although challenging in the prevailing context.
The public health laboratory assessment was positively welcomed by all MECIDS members. The report gave them new knowledge about the situation in the other partner countries. As a result of the baseline assessment, the Palestinian MOH was more aware of the need for clear protocols to be followed in their laboratories and the importance of security checks on personnel entering the labs. They have provided training on protocols and have installed a system of security checks. They have no resources to implement the recommendations made on the requirements for the establishment of BSL3 laboratories.
MECIDS members acknowledged the role of the project in identifying the priority public health topics of common interest for future work. They also confirmed that the level of cooperation across the health ministries has moved to a new level and that the simulation exercise added value to the collaboration process. All members recognize the more strategic benefits achieved through a deeper level of cooperation around academic collaboration but recognized the political obstacles involved.
Palestinian members felt that the project has opened up and consolidated the existing professional network. Jordanians felt that while the East Mediterranean Region Office of the World Health Organization already provided a platform for Palestinian-Jordanian regional cooperation, the simulation exercise helped formalize the working process for regional biosafety work, including the Israeli MOH.
As for Sustainability, there has been a positive transformation in attitudes towards the ‘other’ overall and of skills. Yet, participants feel that more work needs to be carried out with the relevant institutions in their countries to create a more conducive and sustainable utilization of the new skill.
All participants at senior levels agreed that it was important to continue working with personnel from other health ministries. At the level of lab technicians, cooperation remains at the theoretical level at this point, but there is a clear desire to keep in touch and willingness to work on common issues. All data shows that attitudes have improved at all stakeholder level towards increased cooperation and collaboration.
Indicators: Almost all participants (95%) also expressed a readiness to work with each other on common issues.
At the end of the project, training participants had gained significantly higher levels of understanding of and positive attitude to the other. The average score at end line for all training participants (junior lab technicians) combined was 59.7, which is a 10.5 percentage point increase over the baseline of 49.2.
The goal indicator - percentage of participants who agree that they have a better understanding of the other which helped to change their attitude positively –showed a significant increase overall of 10.5 percentage points over the baseline figure of 49.2 for junior lab technicians and a 9.4% increase over the baseline figure of 58.1% for those Senior lab technicians who were trained as trainers (TOTs).
Almost all participants (95%) also expressed a readiness to work with each other on common issues.
While all other findings and indicators were mostly positive, the results indicate a major area of improvement and further focus in the future. The Sub Intermediate Result 1.1 Indicator – health officials licensed by each country’s Ministry of health have increased conflict mitigation skills – showed an overall decrease of 7.7 percentage points from the baseline figure of 45.4%, although more Israelis rated their skills higher at end line by 6.8 percentage points. In general, the focus on this element was minimal for reason explained throughout the report, which calls for either a change of objectives or a change in approach as the recommendations suggest.
Conclusions and Recommendations
The Regional Health Cooperative project is fundamentally a very successful project. All project results were fully achieved with the exception of increasing CM skills, which was only partially achieved. In terms of meeting it objectives, the project succeeded in harmonizing biosafety levels at the human resource across the three countries. The process of harmonizing these levels in terms of other resources – equipment and running costs – has been pushed to the next step through the lab assessments carried out in the three countries by the project. The project also succeeded in building and further developing individual and institutional cooperation on biosafety in the three public health sectors through the joint training activities carried out and through the desk top exercise with MECIDS members, which brought additional value to the ongoing regional cooperation.
The goal of building professional and personal understanding among senior public health officials and technicians in the three countries was reached. In the deteriorating political context and climate, this is a remarkable achievement and attests to the common values held by public health practitioners and the sense of belonging to one epidemiological family.
SFCG staff felt that the project was the most successful of all the MECIDS programs to date because of its systematic planning process and monitoring and evaluation procedures in place. SFCG felt that their ability to plan the project systematically were the direct result of the significant level of USAID financing and USAID requirements for systematic M&E protocols.
The evaluation has the following six recommendations to make for future regional programming in the health sector:
Integrate CM skills into technical trainings, present the skills CM skills and themes as communication skills.
As the CM trainers attested, CM requires much more time for positive outcomes. It is a process which has its own dynamics and used in conjunction with technical training, particularly in a tight training framework, tends to distract rather than add value. This requires further thinking and decision on how to best approach it (directly and upfront, indirectly or even separately).
There is clearly a role for the concept of ice breakers and empathetic listening techniques to be introduced within the broader training process, but they could be presented as communication skills for training rather than CM. These ‘soft’ skills can be delivered by the same trainers who deliver training on technical skills to avoid the potential conflation of CM with the broader political issues. SFCG staff also felt that the ‘light touch’ of the CM modules did not have a significant impact on the undoubted successful outcomes of the project itself. It is difficult to make a distinction between CM and the implicit ‘normalization’ involved because of the conflation of military occupation with ‘conflict’.
Build in sustainability to program activities by providing a broader platform or encouraging MECIDS to establish a broader platform to maintain connections and follow up with all trainers and trainees on a regular basis to track what worked well and was applicable and what could be improved in the training and networking activities.
When participants return to their own institutional environments, the application of what they have learned and changes in their perceptions will not always be smooth. It is important that some kind of platform is established for participants to share their successes, reflect on aspects which are not working well, and discuss particularly useful biosafety messages they have used in their work environments. Such a platform will be used by those participants who are keen to maintain momentum through the opportunities that networking allows. Furthermore, for participants who agree to stay in touch and be available, the next step in consolidating professional networking post training is to provide the tools for participants to engage. An essential link in this is a simple contact sheet which can be presented to all participants at the beginning of the workshops.
Establish clear protocols for the training workshops and enforce as necessary
It is essential that clear instructions in terms of the language to be employed in the training groups are established. Enforcing these instructions must be constantly attended to in order to provide fair and equitable access for all participants to the knowledge being shared.
Agreements on all other the protocols for the workshop, for example the turning off of mobile phones, clarifications of all points requested and a slower pace of delivery so that language competencies can be equalized, must be discussed and approved by participants at the start of all workshops. Attention to these details help to operationalize the inclusive and participatory principles guiding the project design and implementation.
Ensure equity in treatment for all participants and sensitivity to different cultural customs.
Getting this right in such a complex logistical and political context is daunting. But if it is not right, then attitudes of participants to the ‘other’ can be significantly affected. It is worth additional investment in this part of planning and implementation, even if it proves time consuming. By being responsive to requests, explaining changes to the plans and being more proactive and structured in choices of down time activities, participants will feel they are all of equal value.
De-escalate potential issues up front at the right level
For example, if a senior Israeli or even the project officials had clearly explained (or possibly apologized) to the Jordanian participants at the beginning of the workshop for their visa refusal and their perceived rather harsh treatment in the Israeli Embassy in Amman, much of the understandable resentment could have been dissipated. While it may be a common experience for Palestinians, it is not the same matter for Jordanians, who have more positive expectations as the two countries have a peace treaty.
Building on the baseline laboratory assessment
The benefits to the participating MOHs from the base line assessment are not yet clear. A repeat assessment (re-audit) would be helpful to understand how the three governments reacted to the recommendations would be helpful in understanding what follow up actions could be taken in terms of supporting any funding required for additional equipment or training for staff.
Table of Contents
List of tables and charts 3
List of acronyms 4
Executive summary 5
Project Information 11
4. Findings against evaluation criteria 17
41. Effectiveness 17
411. Networking opportunities 17
412. Achievement of project results 18
42. Relevance 20
421. Public health lab assessment 21
422. Improvements to health sectors 21
43. Sustainability 22
431. Improving collaboration and synergies 23
432. Transforming attitudes to support increased 23
Cooperation and collaboration
433. Transforming attitudes towards the other 23
Project Indicators 23
51. Goal Indicator 23
511. Junior lab technicians 24
512. Analysis of findings 26
513. Senior lab technicians 27
514. Analysis of findings 29
52. Sub-Intermediate Result 1.1 Indicator 30
521. Analysis of findings 32
53. Summary of end line results 33
6. Conclusions and Recommendations. 34
Annex 1 Methodology
Annex 2 Data gathering tools
Annex 3 List of people interviewed
The Regional Cooperative Health Initiative (RCHI) is implemented by Search for Common Ground (SFCG) through the mechanism of the Middle East Consortium on Infectious Diseases (MECIDS) which is administered by SFCG office in Jerusalem. The RCHI is a technical training and professional networking opportunity for public health officials and technicians at various levels (junior and senior lab technicians, senior managers) in Israel, Jordan, and Palestine.
Made possible by the generous support of USAID, the main goal of the RCHI was to build professional and personal understanding among junior and senior public health officials and technicians in the Israeli, Jordanian and Palestinian public health sector. This goal was intended to be achieved by building institutional and individual cooperation among the Israeli, Jordanian and Palestinians public health sectors on biosafety and, at the same time, harmonizing biosafety levels in Israel, Jordan and Palestine. Project activities included a joint training-of-trainers for 24 senior lab technicians in biosafety techniques and conflict mitigation (CM) skills, a joint training of 60 junior lab technicians in biosafety and biosecurity to Biosafety Level 3 (BSL3), an assessment of biosafety needs and standards for each country, and a scenario simulation of a biosafety outbreak with senior public health officials of the three countries. Three meetings of the MECIDS Board were also included as project activities although because of significant re-scheduling of project activities during implementation, only two Board meetings were held.
The RCHI programme ran from September 17th, 2015 to March 31st, 2017. A baseline report which assessed the current capacities and practices of BSL-2 and BSL-3 Laboratories regarding biosafety and biosecurity in the West Bank, Israel and Jordan to benchmark training requirements was carried out in April 2016 before the training programs were designed. Trainings in Biosafety and Biosecurity for Level 3 Laboratories were held in Jerusalem in May 2016 for 23 senior lab technicians from all countries (nine from Palestine, seven from Israel and seven from Jordan). Eight of these senior lab technicians were selected as trainers for the subsequent two rounds of training for 59 junior lab technicians (20 from Palestine, 20 from Jordan and 19 from Israel) held in Cyprus in November 20162 Trainings were also conducted for 19 MECIDS members and senior public health officials ( five from Palestine, seven from Israel and seven from Jordan) in Cyprus in November 2016 on how to manage a cross border outbreak of an infectious disease and develop a common action plan for use in the event of public health emergency. Two board meetings for MECIDS members were also held during the two training periods, one in Jerusalem and one in Cyprus.
At the heart of the Israeli-Palestinian conflict is a dispute over land and borders between France, England and Russia at the beginning of the twentieth century as the Turkish Ottoman was dismembered. As a result of agreements concluded at the end of World War 1, Palestine became a British protectorate. After World War 2, the United Nations General Assembly decided in 1947 on the partition of Palestine into Jewish and Arab states, with Jerusalem to be an international city. The plan, which was rejected by the Palestinians, was never implemented. In three successive wars (1948, 1967 and 1972) Israel made massive territorial gains including the West Bank and Gaza, the Syrian Golan Heights, and the Egyptian Sinai up to the Suez Canal. The principle of land-for-peace that has formed the basis of Arab-Israeli negotiations is based on Israel giving up land won in the 1967 war in return for peace deals recognizing Israeli borders and its right to security. The Sinai Peninsula was returned to Egypt as part of the 1979 peace deal with Israel. Jordan signed a peace treaty with Israel in 1994 after the Oslo Accords were signed between Israel and the Palestinians. The Oslo Accords, under which the Palestinian Authority was created in 1994, were intended to lead to a final negotiated settlement between the parties. However, a final political settlement between Israel and Palestine has not materialized.
One consequence of the Arab-Israeli conflict is a lack of cooperation - and sometimes outright hostility - between the Israeli, Jordanian, and Palestinian governments. In the public health sector the conflict has created an environment of mistrust among professionals in the three Ministries of Health, leading to lack of harmonization of information, standards and professional coordination. This presents a public health risk to the three populations.
In this context, the Middle East Consortium on Infectious Disease Surveillance (MECIDS), administered by Search for Common Ground (Search) has consistently worked to improve cross-border efforts on public health between the Israeli, Palestinian, and Jordanian governments over the past thirteen years. MECIDS has served as the only mechanism through which key players in the participating Ministries of Health have conducted joint trainings, shared information, and coordinated cross-border responses on a variety of infectious disease issues.3 MECIDS was formed in 2003, when health professionals from the Ministries of Health and academia of Jordan, Palestinian Authority and Israel, convened together by the US Search for Common Ground, to fulfill the goal of facilitating trans-border cooperation in response to disease outbreaks. In January 2007, MECIDS formed an Executive Board with rotating chairmanship to each country each year. The first targets for MECIDS work were food-borne diseases and avian and pandemic influenza. Food imports and exports from the three countries may provide means for transmission of foodborne diseases in the whole region. Despite the volatile shifts in the political situation MECIDS has responded to the Avian flu outbreak in 2005 through collaborative diagnostic techniques and culling of poultry, to the swine flu epidemic in 2009 through containment and control, and put in place a common action plan for managing any future flu pandemic in 2008.
The RCHI is based on the following theory of change: If large numbers of public health professionals from Israel, Jordan, and the Palestinian Authority are brought together in the context of capacity-building activities on biosafety (trainings, scenario exercises) which will benefit them professionally and serve their societies, then they will be more willing to meet their peers on the other side of the conflict in other contexts. This is because currently, many Palestinians and Jordanians avoid contact with Israelis that might be deemed ‘normalizing’ 4and could lead to negative actions by their fellow citizens against them.
If these activities build up the biosafety capacities of health laboratories licensed by the three Ministries of Health to the same level, then it will also build confidence among professionals in the laboratories where standards were lower and will enable cooperation that was not possible before. Under these conditions, if we introduce capacity building and conflict mitigation skills, then public health officials will be more willing to meet, build trust, resolve tensions, and ultimately improve coordination among the three public health sectors.
This program logic is represented below as the Results Framework5 below, Table 1.
Goal: Professional and personal trust among senior public health officials and laboratory technicians in the Israeli, Jordanian, and Palestinian public health sectors is built.