FHI - I. Capacity Building and Management Issues

E. Capacity Building

As summarized in the process data table under section C, AIDSCAP/Ethiopia, in collaboration with IAs, conducted training workshops for health care providers from the 20 STD clinical sites, Regional Health and Education Bureau project staff (in the focus sites), teachers, school directors, guidance counselors, and NGO project coordinators. AIDSCAP also organized a study tour for 11 government, NGO, and CO staff to Zimbabwe and Tanzania to share experiences with HIV/AIDS prevention programming. Long and short-term training was also organized for 3 MOH staff members. IA project coordinators and other staff participated in regional and international conferences. The exposure to up-to-date information on HIV/AIDS prevention through training, new experiences from the different HIV/AIDS intervention, and challenges ahead has helped project staff strengthen HIV/AIDS prevention activities in their respective projects.

The CO, government ministries, and NGOs conducted baseline surveys using locally hired consultants, thus building indigenous research capacity, particularly in FGDs and in-depth interviews.

It is important to mention the problem of high staff turn over, which is cited as a constraint to project implementation. This problem is somewhat alleviated if the trained staff remain within a similar area of work even though their geographic location may change.

The country office hired consultants to conduct three assessments relating to capacity building within the AIDSCAP/Ethiopia activities throughout the life of the project. In consultation with the USAID Mission, AIDSCAP's Africa Regional Office, and AIDSCAP HQ, the assessments focused on evaluating the capabilities enhanced or strengthened within the IAs in terms of a transfer of technical skills, training effectiveness, and the success or failure of IEC materials. The three assessments were conducted between October 25, 1996 and December 15, 1996, and covered selected institutions and organizations at central, regional and ten clinical sites located in Regions 1, 3, 4, 14, and the Southern Nations Nationalities People's Region (SNNPR). The researchers interviewed 31 project managers (or their deputies) for the organizational and training assessments. For the IEC materials assessment, they conducted 25 FGDs with target audiences to determine access, exposure and comprehension of materials, and 26 in-depth interviews with IEC program coordinators to examine various procedures employed in the process of materials development and distribution. In addition, the researchers interviewed 39 people who had participated in training workshops organized by AIDSCAP. Data collection instruments were provided by the AIDSCAP project office and modified slightly by the researchers in collaboration with country office staff. The following sections summarize key findings.

The researchers cautioned that interpretations of the results of these assessments be made keeping in mind that,

"In the absence of a baseline survey, one factor that has to be considered is the difficulty to isolate the support of AIDSCAP from the support provided by other donors which have contributed to the strengthening of the administrative, managerial, or technical areas of the IAs. Any intervention is likely to affect how an organization functions, in multiple ways so that it becomes difficult to isolate the specific effects of specific interventions."

In addition to this point, the researchers also emphasized the critical role that the organizational environment of government institutions and NGOs played in the implementation of AIDSCAP/E interventions. The upheaval caused by the decentralization of the MOH/DAC and the increased staffing at regional levels came at exactly the same time as AIDSCAP/Ethiopia was disbursing its first funds, as described in more detail earlier in this report. Assumption of the same magnitude of responsibilities by the regional bureaus as the central DAC had assumed during previous years, did not happen as envisioned. AIDSCAP's support during those difficult transition times was seen by most respondents as fortunate and supportive.

1. Rapid Organizational Assessment

The assessment used the Comprehensive Key Informant Interview Questionnaire (CKIIQ), developed at AIDSCAP HQ, to collect data about technical skills building, organizational and management skill building, organizational systems development, and networks and global learning enhancement. Assessment findings are summarized below.

The number of professional and support staff has increased in all government offices and NGOs participating in AIDSCAP interventions, except in the central-level MOH where there have been substantial decreases in staff (due to government restructuring) and in the central-level MOE where staffing has remained the same. The increase in the number of staff at regional levels and in NGOs is attributed to restructuring and decentralization of government organizations and expansion of services for NGOs.

As a result of the input from AIDSCAP, MOH and other donors, the level of expertise at RHBs was considered by program managers to have shown improvement at the end of the AIDSCAP project period Areas that have shown improvement include STD management, training/education, materials development, and project planning.

The overall level of expertise or capability of staff at five health centers has shown improvement in the areas of training/education, interpersonal communication, STI management, resource leveraging, project monitoring, mass media, and condom social marketing. The Awassa health center was the only one not showing improvement in project monitoring, mass media, and CSM.

The capability of staff in all of the nine NGOs included in the study has improved, especially in BCC and gender planning in one NGO, due to AIDSCAP's and other donors' input. STI case management, the Focus Site Intervention Teams, and health education by Anti-AIDS clubs in schools were considered to be sustainable without AIDSCAP funding.

Respondents especially mentioned technical skills building in terms of training in STI case management for health workers at clinical sites, and most notably, the introduction of syndromic management and BCC materials related to STIs. Other respondents cited major improvements in technical skills for developing IEC materials, counseling, training, and designing communication interventions.

Improvements in organizational and management skills were identified by respondents as resulting from AIDSCAP support, especially in project monitoring, project evaluation, MIS, and equipment/supply inventories. No changes were reported in the financial management of all government institutions and NGOs, except SYGA which developed an accounting manual after the start of the AIDSCAP project.

In one NGO (CRDA who collaborated with MSIE), gender planning and development was introduced as a result of AIDSCAP's provision of training to the program coordinator. CRDA, FGAE, NACID, and CCF were the only IAs undertaking gender-related activities during the AIDSCAP project. The assessment concluded that "Most of the NGOs and government institutions have no skills or only minimal skills in the areas of gender planning and development, research methods, focus group methods, KABP survey methods, and interviewing techniques."

2. IEC Materials Assessment

The 25 FGDs were carried out in all four focus sites and in Addis Ababa, divided equally between the five locations. The five groups included 151 participants representing MPSCs, out of school youth, factory workers, high school students and teachers. During each discussion, participants were shown 19 different IEC materials used by IAs working under AIDSCAP/E. The discussion also included comments, reactions and views of participants on audiovisual materials (video films and audio cassettes) produced during the project.

Most respondents among different target groups confirmed that they have been exposed to materials produced through the support of AIDSCAP/Ethiopia. These materials include posters, leaflets, flip charts, video films and calendars. In general, Anti-AIDS club members, MPSC female group leaders, AIDS communicators, and youth peer educators had more exposure to the IEC materials than others in the target populations. Specifically, the former group had access to more than three types of materials while the latter group (the target group members) had access to not more than two materials, mainly posters and leaflets.

The primary source of IEC materials for MPSC females, and in school and out of school youth are reported to be PHEs, whereas for factory workers, AIDS Communicators and factory clinics are the main source.

Most of the study participants confirmed that they have transmitted the IEC materials and messages to others, except in Mekele School where teachers said that since there was no directive telling people to transfer the materials after reading, they did not do so.

All participants confirmed that they understood the messages and concepts to which they were exposed. Most FGD participants considered the IEC materials to be useful and appropriate. Some materials and messages were mentioned as being especially timely and appropriate, such as the emphasis on "ways in which AIDS cannot be transmitted," described by FGD participants in Nazareth and Awassa, which came at a time when rumors were circulating that HIV could be spread by "chickens eating used condoms" prior to the chicken being consumed as food by humans:

"There was a rumor that was widely circulated among the community in Nazareth and its surroundings that people have seen chickens swallowing used condoms in the area. Consequently, people stopped eating chicken and eggs. After some time, this popular poster was distributed in the area and it was after the dissemination of the poster that most people especially those in the surrounding rural areas, started to eat chicken and eggs [again]."

High school students in Nazareth cited leaflets and booklets with pictures as being the best IEC materials. They considered the poster "Marriage is our shield -- We are not afraid of AIDS" produced for rural people as least acceptable for these reasons:

"The pictures on the poster look like peasants and are not attractive to urban dwellers in general, particularly to young people in Nazareth. For me, they are comparable to the warning written on the road side walls which prohibits pissing but is hardly respected."

"Let us provide care for AIDS patients," prepared by Region 14, was least accepted by MPSC females because the message and the picture illustrated were not considered to support each other. The flip chart produced in Bahirdar by the RHB was least liked by out of school youth because it was too bulky to carry around. In general, video films were especially well-liked by all FGD participants. For instance, the video films entitled "Megedegnawa mushira, Alem Bekagn" and "Life in flame" were appreciated by out of school youth because of message clarity, language appropriateness, and persuasiveness.

The booklet "Know and Act" produced by the MOE for distribution to high school students was considered least acceptable by FGD participants from Awassa, Mekele and Nazareth. One student commented:

"I was given a booklet entitled "Know and Act" at school. This booklet was prepared in English and distributed to us assuming that high school students can read and understand the message. I was in Grade 9, but could not read and comprehend the message in English. So I just kept it at home."

The findings of the IEC materials assessment study were triangulated by many of the findings of the BCC Lessons Learned research, also conducted in the fall of 1996, and described above.

3. Training Assessment

AIDSCAP/Ethiopia conducted 19 training workshops and seminars between 1993 and 1996, training a total of 162 people out of the 334 personnel working for the IAs. The assessment included interviews with 31 program managers and 39 training workshop participants, including physicians, health officers, nurses, sanitarians, pharmacists, a health educator, two lab technicians, a health assistant, one sociologist, four high school teachers, one FP educator, one health education coordinator, one counselor and one administrative clerk.

The training workshops conducted by AIDSCAP/Ethiopia can be classified broadly as skills and/or retraining. The purpose of this assessment was to see whether the training given by AIDSCAP in managerial and technical strategic areas has facilitated improved implementation of the project. The following points summarize the researchers' findings:

95 percent of the respondents confirmed that they have acquired new knowledge and skills from the AIDSCAP training and almost all of the participants have used the knowledge and skills they acquired in their day to day activities;
95 percent of the trained respondents confirmed that they have transferred the knowledge they acquired to their colleagues;
92 percent rated the training as good and 82 percent rated the method of training as very good;
90 percent confirmed that the content of training is relevant to their fields of operation;
Short duration of the training was mentioned by about 54 percent of the respondents as a shortcoming of AIDSCAP training.
Suggestions given by participants to improve future training include: increase duration of the training, provide regular follow up of and support to the trainees, continue training to overcome high staff turnover, improve teaching methods through building facilitators' capabilities, provide broader training coverage from central to the grass root levels, and provide it to more sites than just the STI clinical sites.

Types of training considered to be most useful include (mentioned in decreasing frequency): effective approaches in BCC, STI management, monitoring and evaluation, and interpersonal communication. The majority of program managers interviewed stated that all the training workshops were useful. Five mentioned as least useful an orientation on USAID's accounting system (because it was not relevant to their work). One mentioned monitoring and evaluation (because they did not use the knowledge/skills acquired), and one mentioned the materials development training (because the training was not used since materials development was done at a higher level).

The researchers made the following comments in their conclusions underscoring the importance of these activities to AIDS prevention in Ethiopia:

"The implications of these findings including the suggestions given by the training participants clearly show that such training has to be sustained and a mechanism created to ensure its continuity and replication at all levels.

"In view of the decentralization process and inadequate financial resources, the need for capacity building at the central, regional and woreda levels is more acute now than ever before. Consequently, the training workshops conducted by AIDSCAP are extremely needed at this juncture in order to consolidate the gains achieved so far and contribute to further capacity building on a national scale. Most of the suggestions given by the training participants could materialize and the replicability and sustainability of project activities could be ensured if capacity building measures are fully supported by USAID/AIDSCAP, and the training process institutionalized at the central and regional levels.

"This could only be achieved if the support of USAID/AIDSCAP were to continue for some time to come and its focus redirected to concentrate more on providing skills training for the staff of collaborating agencies."

F. Management Issues

The AIDSCAP/Ethiopia country office was established in January 1993 to implement USAID assisted HIV/AIDS/STIs prevention projects. Following the recruitment of the Ethiopian Resident Advisor, project time was spent in preparatory activities for future interventions. The activities at the early stage focused on building relationships with the National AIDS Control Program implemented by the MOH Department of AIDS Control, establishment of the AIDSCAP country office, employment of required staff, putting a communication system in place, purchasing office furniture, equipment and supplies, holding meetings with potential implementing agencies, and conducting preliminary training.

Initially a team of AIDSCAP technical assistants came from headquarters and spent three weeks in Ethiopia developing a one year work plan with local collaborators for implementing the AIDSCAP portion of the STAC project. At the same time, the team was asked to make recommendations for an amendment to extend the project beyond the first agreement which was in September 30, 1994. The whole STAC project thus was extended to run through September 30, 1995. A detailed work plan was then developed outlining activities in the areas of STIs, IEC (for in and out of school youth, and MPSC women and men), condom promotion and distribution (primarily condom social marketing through DKT/PSI), NGO grant support, surveillance, and behavioral research.

The activities from 1993 through Feb. 1994 included the signing of a comprehensive subagreement with the Department of AIDS Control for STI, IEC and surveillance activities, and the initiation of activities for in-school youth with the Ministry of Education. Equipment, supplies and essential STI drugs provided through a WHO letter grant for the refurbishment of STI clinics arrived and refurbishment began to arrive. A major activity was the development of the NGO Competitive Grant program, solicitation and review of proposals and funding of five NGOs. In 1994 different LOAs were also signed for the noncompetitive grants that targeted the various population groups described elsewhere in this report.

During the second phase AIDSCAP started to work more closely with the four focus sites (Mekele, Bahirdar, Nazareth, Awassa and later in 1995 Addis Ababa was included). Monitoring and evaluation systems were put in place to track program activities. In 1994, the second phase implementation plan slowed down due to uncertainties surrounding the Department of AIDS Control and the regionalization and restructuring process that was taking place at the time. DAC was left with only three people out of 57 original members. The AIDSCAP country office had to add a technical project coordinator in order to expedite focus site and clinical site activities and compensate for the downsizing of the DAC.

The STAC-1 project was reviewed in August 1994 by a USAID mid-term evaluation team and a decision was made to terminate the MOH subagreement as of September 30, 1994 because of slow progress in implementation of activities and utilization of funds transferred from FHI/AIDSCAP. The decision also required the return of unused funds to FHI/AIDSCAP/Ethiopia to carry out planned activities through direct disbursement of funds from the AIDSCAP Country Office. In October 1994, the AIDSCAP evaluation team followed the recommendations of the USAID mid-term evaluation and provided managerial and technical directions. The need for additional staff was recognized, taking into account the duties and responsibilities of the country office and the overburdened staff.

Additional staff were employed and the Country Office took over the financial responsibility of the MOH as of October 1, 1994. It was encouraging to see activities in the regions accelerate as of this date. On the other hand, although implementation continued in the regions, the additional administrative aspects of the MOH project impacted upon the technical support that needed to be given to the regions. The Country Office staff had to handle both the financial and the technical parts of the large MOH project. During this time, it was difficult to attend to financial matters with the participants of the training and refresher programs and, at the same time, provide the technical support they needed. Because of the direct disbursement of funds to activities, the Focus Sites found it difficult to believe that the project belonged to them. The sites did not develop a sense of ownership of the project, and AIDSCAP was seen as an NGO implementer of a project rather than an expediter of implementation of their projects.

The collection of Process Indicator Forms was another problem as monthly process data was not easy to obtain from the collaborating agencies, particularly from the Focus Sites. Since most NGOs operate in Addis Ababa, it was not difficult to get the information needed for PIFs. However, obtaining data on time from outside Addis is difficult due to the slow postal system, and also to the reporting systems that move from the regions to the central ministries. The only option was to collect the data from the Regions by telephone and during monitoring visits by CO staff.

The gap in funding between the end of the STAC Project in December 1995 and the beginning of the ESHE Project was a period of uncertainty both for the country office and for the focus sites' Regional Health Bureaus. The country office continued to monitor activities only through telephone contact with the STI/AIDS coordinators. The expectation of the Regional Health Bureaus to continue activities according to their 1996 work plan remained unanswered for quite some time, and this affected activities in the Focus Sites. Although a no-cost extension was allowed from September 1995 to December 31, 1995 for all the implementing agencies, the period of uncertainty covered about six months before activities started under ESHE.

While activities eventually started with Regional Health Bureaus, NGO proposals except for Rapid Response Fund were not approved, and this again raised unnecessary expectations on the part of the NGOs at the beginning of 1996. Since April 1996, activities started under ESHE and because of budget cuts, two staff members had to be let go as of July 1, 1996. The CO then was without a senior program officer after September 1996. Since then, the country office had to implement activities according to the delivery order plan throughout 1996 with one additional STI consultant since September 1996. As had occurred throughout the life of the project, technical assistance continued to be provided by AIDSCAP Regional Office and Headquarters, however, even external TA could not completely compensate for staffing shortages.

Final Report for theAIDSCAP Program in Ethiopia: Country Program Description

Final Report for the
AIDSCAP Program in Ethiopia: Country Program Description