The World Health Organization (WHO) defines voluntary HIV counseling and testing (VCT) as a confidential dialogue between a client and a care provider aimed at enabling the client to cope with stress and take personal decisions related to HIV/AIDS1.

Although the effectiveness of this intervention in changing people's behavior to reduce the risk for HIV infection had been under debate until recently, VCT is already a major component of HIV prevention and care programs of most developed countries and is being promoted in many developing countries2, 3.

The existing literature on counseling and testing is almost exclusively composed of reports on studies that have tried to assess the effects of VCT on behavior, with before and after intervention evaluation as the most commonly used study design3. Very little in the literature addresses issues such as how well the service is provided, how the service is perceived both by the clients and providers, or how cost-effective the service is as provided. This scarcity of information is unfortunate because people affected by HIV/AIDS want HIV counseling and testing services for future planning (including planning for marriage and children), emotional support, medical services, and other referral services4. As such, VCT services require continued, comprehensive (when possible) evaluation to help adapt the service in response to evolving knowledge, client needs, and technology.

This chapter provides general guidelines for evaluating counseling and testing programs. Although the goals and objectives of counseling and testing programs may vary from one country to another and from one program to another, this chapter is intended to serve as a practical reference for service providers, program managers, and those called upon to evaluate HIV prevention programs.

Objectives Of VCT Evaluation

In general, voluntary counseling and testing pursues two interdependent objectives:

• to enable clients to plan and to cope with issues related to HIV/AIDS; and
• to facilitate preventive behavior

As stated above, the specific objectives of counseling and testing may vary based on the needs expressed or identified during the planning of HIV/AIDS prevention and care programs. For example, in the United States, HIV counseling and testing is used for surveillance, promoting behavior change, public education, and referring individuals into treatment and care systems5. In most developing countries, counseling and testing programs are essentially designed to influence clients' risk behavior and facilitate social and medical support for clients who test positive.

For an evaluation to produce results that will inform the design, implementation, and improvement of VCT programs, one must take into account the program's objectives. However, whatever the program objectives are, evaluation activities should address two main areas most relevant for service providers and policymakers:

• Service delivery-How well voluntary counseling and testing is provided
• Program effectiveness-The intermediate outcomes and long-term impact that voluntary counseling and testing may have on the population receiving the service

Once program goals and objectives have been clearly defined, the next critical step is selecting appropriate indicators to monitor and evaluate the VCT intervention. Table 6-1 lists examples of program indicators that may be useful in addressing the different levels of evaluation for voluntary counseling and testing services: service delivery/program outputs, intermediate program outcomes, and expected program impact.

Evaluating Service Delivery And Service Use

Depending on the goals and objectives of the program and the interests of the program managers, an evaluation of service proficiency may cover all aspects related to providing the service or it may focus on one or more specific aspects. The sources of data will also be a function of the aspect of the service to be evaluated. For example, client interviews are an appropriate means to measure clients' satisfaction with the service. Key aspects of VCT services that should be evaluated include counseling and testing protocol adequacy, staff performance, and service accessibility and barriers. Each is discussed below.

Counseling Protocol Adequacy
Counseling and testing protocols may vary from one program to another based on the goals and objectives of the program. However, whatever the approach taken, the VCT intervention must be regularly evaluated to determine whether it is provided in accordance with the pre-determined protocol and whether it satisfies clients needs. Results can be used to improve the quality of the service provided. Counseling adequacy is defined by the main components and characteristics of voluntary HIV counseling shown in Box 6-1. Questions that must be answered include:

• How well do the counselors follow the counseling protocol?
• Do the clients feel their confidentiality is protected?
• Is risk assessment conducted? If so, how well is it done?
• Is information provided on HIV transmission and risk factors?
• Is a risk reduction plan discussed?
• Is the meaning of the HIV test explained?
• Is the HIV test result clearly given?
• Is emotional support provided?
• Are referrals for medical and social support provided?
• What is the waiting time at the VCT site?
• Is partner notification conducted? If so, how is it done?

An analysis of the answers to these questions will provide feedback to be used in improving the quality of the service provided.

Testing Protocol Adequacy
The testing protocol for a VCT service must be designed to reach maximum reliability and validity in accordance with local conditions, such as the type of equipment available, local HIV seroprevalence, and the resources available to acquire the recommended test kits. The testing protocols used in VCT programs must be examined against the testing strategies for HIV diagnosis recommended by UNAIDS and the World Health Organization to ensure that they are adequate for the local context6. The evaluation of the testing protocol must provide answers to the following questions:

• How consistently is the protocol used?
• How valid is the testing algorithm in terms of specificity and sensitivity?
• How long must clients wait to receive their test result? Are clients comfortable with the waiting period?
• How much does the testing protocol cost?
• Is the testing protocol the most appropriate given local conditions? If not, how can it be improved?

Staff Performance
VCT service requires well-trained and motivated personnel. Regular monitoring of their performance is essential to ensure quality and may help to prevent staff burnout. Focus must be placed on such questions as:

• How well trained are the counselors?
• How well do counselors deliver the protocol?
• Are counselors well informed about other issues relevant to VCT services, including testing technology, options for HIV-positive women who are pregnant, and possible referrals to care and support services?
• How well do counselors meet clients' needs?
• Are counselors appropriately supervised?
• Are the counselors appropriately used?
• What mechanisms are in place to help counselors solve problems and deal with stress?

Service Accessibility and Barriers
It is important to identify factors that affect accessibility and create potential barriers to service use. Parameters to be evaluated include:

• How far must the intended population travel to reach the service?
• Is public transportation to the VCT site available?
• How much does it cost for clients to receive VCT services?

Because clients must pay for medical services in many developing countries, it is essential that cost does not become a barrier to individuals using the service, especially those who might need it the most. When assessing accessibility, it is also important for those who are planning an evaluation to keep in mind that being near to a VCT site does not always guarantee easy access to the service. In fact, in areas where there are strong stigmas attached to HIV/AIDS, proximity can be a barrier to service use because potential clients may prefer to go to a VCT site far away from the sight of their neighbors, who may suspect them of being infected just because they visited a VCT center. In this context, it is also important to assess who is being reached by the VCT site:

• Are those at highest risk obtaining the services?
• Are significant populations or groups not being reached?

Service Use
The extent to which services are used is an important factor in determining the viability of a program. A VCT service with a minimal level of use by the target population is not cost-effective and therefore unlikely to receive support and continued funding, even if it is effective in other ways. An evaluation of service use must answer the following questions:

• Who uses the service?
• How many clients are served?
• Why do people seek the service?
• Do clients complete all procedures involved in using the service?
• Is the level of use sufficient to justify sustaining the service?

Sources of Data
Evaluation data on VCT service delivery and use can be obtained from various sources. Table 6-2, adapted from Coyle et al., summarizes the sources of data that can be used to evaluate different service aspects5.

Service records and the staff of counseling and testing sites are an important source of information for the evaluation of VCT services. The sites' records may be used to collect data on the level of service use, the characteristics of clients attending the sites, reasons for using counseling and testing service, and the testing protocol being used. Standard forms containing relevant information must be developed and filled out on a regular basis by the VCT staff. These can be used later for evaluation purposes. The staff of VCT sites may provide useful information about their perception of the quality of the service provided to clients, possible barriers to service use by potential clients, the level and quality of supervision provided to counselors, but also information on the impact-physical, emotional, and otherwise-that counseling and testing has on its providers. Information on these issues may be obtained through periodic key informant interviews or focus group discussions, if appropriate.

Clients can provide information on all aspects of VCT service delivery and use. These data may be obtained through exit interviews with a subsample of randomly selected consenting clients after a counseling session using a standardized questionnaire that focuses on the clients' perceptions of the quality of the session and the counselor's performance, or their impressions of the VCT site in general (for example, accessibility of the site, perceived barriers to use, organization of the service, ability of the service to meet client needs, length of waiting time, cost of the service). In-depth interviews and focus group discussions with a selected number of clients may be used to collect additional contextual information.

Non-participant observers as well as "professional customers" (or mystery clients) may be used to conduct quality assurance activities through direct observation of the VCT procedures. Direct observation can help in assessing the adequacy of counseling and testing protocols and the adequacy of the counseling and testing actually provided (staff performance). Some practical issues related to direct observation must be addressed before it is used, however. For example, clients may be concerned about the confidentiality of the information they reveal if there is a third person in the room during the counseling session. The presence of an observer may also alter the natural way a counselor interacts with his/her client. Efforts must be made to reassure clients and minimize pressure on the staff being monitored.

Surveys of the population at large, which include former, current, and potential VCT service users, or surveys of selected populations, such as hard-to-reach groups in defined catchment areas of the site, provide complementary information about the performance of the VCT service as perceived by the community. In addition, population-based surveys may help to identify what the population expects from the service as well as barriers to service use. Such surveys will also provide information that can be used to characterize people who use and do not use the service, and identify ways to improve the service and make it more accessible and attractive to the population.

Evaluating Outcomes And Impact (Effectiveness) Of VCT Programs

Effectiveness evaluation of VCT programs aims to determine how well voluntary counseling and testing services have achieved their intermediate and long-term program goals (see Table 6-1). Intermediate outcome indicators may measure the extent to which the VCT intervention has encouraged behavior change among clients and their partners and changes in STI rates as a biological proxy indicator for adopting preventive behaviors. Other important outcome indicators should measure the reduction in stigma of, and discrimination against, HIV/AIDS-affected people in the community. Measures of long-term program impact attempt to determine whether VCT intervention activities have affected the rate of HIV transmission in the community, including mother-to-child transmission. They should also include an assessment of the impact of VCT on societal norms in the community reached by the program. To approach this difficult task and be able to make meaningful inferences on program effectiveness, evaluators must analyze VCT process data together with other types of data that are collected in the catchment area of VCT services. These data include behavioral survey data, HIV sentinel data, and ethnographic research data.

A recent multicenter randomized trial conducted by the AIDSCAP Project of Family Health International and UNAIDS/WHO in three developing countries has demonstrated the effectiveness of VCT in changing sexual behavior of those counseled and tested7,8. However, although VCT has demonstrated its effectiveness in changing risk behavior in these selected study sites, VCT program managers still must determine whether the VCT service they provide makes a difference for those who receive it. An effective counseling model in one community may not be as effective in another community. Different models may have to be tried to identify the most effective one for a given setting. The following descriptions of different outcome and impact evaluation approaches show how the effectiveness of particular VCT services can be assessed.

Randomized Design
A randomized controlled design-by far the most rigorous way to measure VCT effectiveness-may be used so long as it is assured that participants in the control group also receive a beneficial intervention (see example in Box 6-2). Different VCT protocols may be tested using this design. For example, same-day testing may be compared to the standard protocol requiring the client to return in a week or two for the test results. Or, a two-session counseling protocol (pre- and post-test counseling) may be compared to an open protocol in which a client uses as much counseling as needed. Counseling and testing programs may also be compared with other prevention interventions. Given the high cost of this experimental design and the scarce resources in most developing countries, program managers must carefully weigh the relevance of this design before embarking on a randomized controlled intervention. It should be used only when there is an important conceptual question to be answered that will have regional or international significance. In reality, few programs will ever conduct such trials because of their expense and methodologic complexity.

Pre-Post Intervention Client Surveys
With this approach, a random sample of clients seen at the VCT site(s) is selected to be followed for a given time period. A standardized behavioral survey questionnaire is administered to the selected clients before they receive any intervention and the same questionnaire is administered to them some time (1, 3, or 6 months) after the intervention. To enhance this evaluation approach, data on STI status should be collected from the clients at intake and follow-up to support the survey findings. Although less expensive and complex than a randomized controlled trial design, such surveys are not easy to conduct in resource poor settings and the results are often considerably biased due to substantial follow-up losses. Moreover, a pre-post intervention design with no comparison group does not allow evaluators to control for behavior-modifying effects that are unrelated to expected intervention effects (see Chapter 1, "Conceptual Approach and Framework for Monitoring and Evaluation," for more information on this issue).

Nevertheless, the evaluation of the AIDS Information Center in Kampala, Uganda, is an example of this approach4. Another example is the work by Kamenga and colleagues in Kinshasa, Congo (former Zaire)9, where married couples were interviewed on sexual behavior before receiving voluntary HIV counseling and testing. Discordant couples (one partner HIV-positive and the other HIV-negative) were then followed and assessed monthly for behavior change (questionnaire) and STI incidence (laboratory testing). Figure 6-1 shows some of the outcomes from this work. However, as noted above, these results have to be interpreted with caution because of the inherent methodological weakness of study designs with no comparison groups.

Conclusion

There is now increasing support for expanded programs of voluntary counseling and testing to enable people to cope with issues related to HIV/AIDS, to encourage preventive behaviors, and to facilitate access to care and support services for people who test positive for HIV. To ensure continued quality and inform programmatic improvement, evaluating VCT services must be an ongoing process that is integrated into the implementation of the service from the beginning. Evaluative activities will be determined on the basis of program objectives and the available funds for evaluation.

For practical and operational purposes, the evaluation of VCT interventions should focus on key service aspects, such as service use, the adequacy of counseling and testing protocols, staff performance, and service accessibility, and should use complementary sources of information that provide different perspectives on the various service performance aspects. These sources include VCT staff and service records, client surveys, direct observation of VCT service provision, and population surveys in the community reached by the program. Special emphasis must be given to ensuring the confidentiality of sensitive information revealed by clients or VCT staff. The data collected must be analyzed and used to ultimately provide feedback to all interested parties at different levels (from the staff to the central authorities) and the methodology used must be carefully selected taking in account program priorities and available resources.

Program outcomes related to behavior change, stigma reduction, and community support should be assessed periodically to determine the extent to which voluntary counseling and testing services have achieved their intermediate program goals and objectives. Measures of the long-term program impact should include trends in mother-to-child transmission of HIV in women of childbearing age because voluntary counseling and testing services play an essential role in interventions designed to reduce this mode of HIV transmission.

References

1. WHO/GPA/TCO/HCS/95.15. Counseling for HIV/AIDS: a key to caring. World Health Organization; 1995.
2. De Zoysa I, Philips KA, Kamenga MC, et al. Role of counseling and testing in changing risk behavior in developing countries. AIDS 1995;9(Suppl A):S95-S101.
3. Higgins, DL, Galavotti, G, O'Reilly KR, et al. Evidence for effects of HIV antibody counseling and testing on risk behaviors. JAMA 1991;266(17):2419-2429.
4. Campbell CH Jr, Marum EM, Alwano-Edyegu MG, et al. The role of HIV counseling and testing in the developing world. AIDS Educ Prev 1997;9(3Suppl):92-104.
5. Coyle S, Boruch R, Turner C, editors. Evaluating AIDS prevention programs. Washington (DC): National Academy Press; 1991. p. 15-32.
6. Joint United Nations Programme on HIV/AIDS (UNAIDS) and World Health Organization (WHO). Revised recommendations for the selection and use of HIV antibody tests. Wkly Epidemiol Rec 1997;72(12):81-87.
7. The Center for AIDS Prevention Studies. The voluntary HIV counseling and testing efficacy study. Final report. Arlington (VA): Family Health International; 1998.
8. The Voluntary HIV-1 Counseling and Testing Efficacy Study Group. Efficacy of voluntary HIV-1 counseling and testing in individuals and couples in Kenya, Tanzania, and Trinidad: a randomized trial. Lancet 2000;356(9224):103-112.
9. Kamenga M, Ryder R, Jingu M, et al. Evidence of marked sexual behavior change associated with low HIV-1 seroconversion in 149 married couples with discordant HIV-1 sersotatus: experience at an HIV counseling center in Zaire. AIDS 1991;5(1):61-67.