Sexually transmitted infections (STIs) are major causes of reproductive morbidity and mortality. Because many of the widely prevalent STIs also facilitate transmission of human immunodeficiency virus (HIV), STI control has been recognized to be an effective means of reducing HIV transmission. The proportion of new HIV infections attributable to STI cofactor effects–the population-attributable fraction–is larger in populations or subpopulations with HIV infection in a growth phase than in populations where the HIV epidemic is mature. In keeping with the rest of this Handbook, this chapter considers evaluation of STI control programs in the context of HIV prevention, with reduced prevalence and incidence of STIs as a desired goal contributing to reduced HIV transmission at the impact level. In this context then, monitoring and evaluating STI programs has two goals. The first is to ensure that STI services are proficient (both clinically and with respect to coverage) in managing and preventing STIs. The second is to obtain proxy biologic measures for both the potential risk of HIV infection spread in a population and the impact of HIV prevention interventions. The relationship between HIV and STIs is complex, however, and the use of STIs as a proxy for HIV is discussed in greater detail later in this chapter.

The term "sexually transmitted infection" (STI) refers to infections that may or may not cause symptoms and recognizes the asymptomatic and sub-clinical nature of many of these infections. The term "sexually transmitted disease" (STD) refers to infections with sexually transmitted pathogens that cause recognized symptoms or clinical signs in individuals. Throughout this chapter and the remainder of the Handbook, the term STI will be used except in specific instances when referring to symptomatic individuals.

Much more experience exists in monitoring and evaluating STI control programs than with most other areas of HIV prevention and care. This reflects the fact that STI control has been a disease control strategy in its own right and the fact that STIs were identified almost a decade ago as a key component of HIV prevention interventions. In this chapter, we provide general guidelines for monitoring and evaluating STI control programs and for STI surveillance. While the specific target populations, target STI pathogens, and the mix of approaches and strategies for STI control will vary from one country to another and from one program to another, this chapter is intended to serve as a practical reference for service providers, STI and HIV program managers, and others called upon to evaluate STI control programs.

STI Control Considerations

Effective STI control requires a coordinated series of interventions at both community and clinic levels to facilitate access to services, to ensure that services are appropriate to the needs of those at greatest risk, and to ensure that the services are of high quality. An explanatory model of STI transmission dynamics and barriers to prevention and control should thus guide both the design and evaluation of interventions. Two commonly cited models are helpful for illustrating elements that should be addressed in evaluation. Health systems models (Figure 4-1) take as a starting point the subset of the population with symptomatic STIs and analyze barriers to identification and effective treatment. They are most useful for pointing out areas where services could be improved to maximize the proportion of STI cases that are effectively managed and rendered non-infectious. Limitations of such models are their focus on formal health care services and their treatment of populations as homogeneous with respect to STI transmission dynamics within a population.

Transmission dynamics models (Figure 4-2) address questions about which subgroups of the population are most at risk of acquiring and transmitting STIs. Such models better explain what occurs at the community level with STI transmission than do health system models and can be useful for orienting interventions to those who can most benefit from them and have the largest impact at the population level. While less informative about needs at the service delivery level, analysis and evaluation within this framework is better suited to pointing out significant gaps (such as poor access to services for those at greatest STI risk) that could have important implications for STI control.

Using both of these models together, an explanatory model for designing and evaluating STI control efforts can be constructed that considers aspects of coverage as well as proficiency of individual service elements. With information from formative research, such models can be adapted to fit local conditions and contribute to effective intervention design. Once an intervention is designed and program goals and objectives are clearly defined, appropriate indicators to monitor and evaluate the STI control effort can be selected. Table 4-1 lists illustrative indicators and data sources that may be useful in addressing the different levels of monitoring and evaluation for STI control programs:

• process evaluation, including inputs, routine data collection on activities, and special study output information; and
• effectiveness evaluation, including intermediate program outcomes and expected program impact

Monitoring And Evaluating STI Control Programs

Formative Evaluation
As a first step in the program development cycle, information should be collected to describe the current situation, identify needs, and orient an effective response. A situation analysis should consider available data on STI prevalence and incidence, capacity and quality of existing

services, current use of those services, as well as access and acceptability issues. The following are some components of a situation analysis for a program seeking to institute or improve STI interventions:

• review of existing STI data;
• community mapping to identify priority areas for interventions and availability of formal and informal sources of care;
• health facility audit to assess availability of prevention and treatment supplies;
• assessment of the quality of STI case management to define training and support needs and establish a baseline;
• key informant interviews in the community to better understand issues related to commercial sex or other high-frequency STI transmission dynamics; and
• focus group discussions with sex workers, clients, and "gatekeepers" to identify
• structural obstacles impeding safer sex and access to services, as well as acceptability of current services

Information from the situation analysis should be synthesized and fit into the explanatory model that describes the dynamics of STI transmission in the community and identifies limitations of existing services. Potential biases in the data should be recognized, however. The most easily collected data are often obtained from formal public-sector health care facilities. Results may be subject to bias when services do not reach the people at greatest risk in the community. Formative research should take a wider community perspective to avoid missing population groups important to STI transmission. The meetings and interviews that comprise formative research activities should also be seen as important opportunities to promote trust and participation of key target populations.

Process Evaluation: Inputs and Outputs
Attention to service delivery through monitoring program inputs and outputs, including service quality and coverage, is crucial for program management. These indicators describe how efficiently program activities are carried out and many, although not all, are collected as part of implementing program activities. All important program activities should be monitored and should be linked

to indicators. One of the most important purposes of monitoring process indicators is keeping services on track. Indicators should be reviewed with project staff on a regular basis to improve management, identify problems, and motivate performance.

Examples of process evaluation indicators and possible data sources for different types of STI program activities are presented in Table 4-1. Sources of data for input-level process indicators are generally obtained from routine monitoring records of service implementation as well as

program records on training and supervisory activities. Similarly, data for many output-level process indicators include simple routine reporting instruments such as clinic registers, log books, referral slips, and program records of activities.

At regular intervals, however, it is important to take a closer look at aspects of quality and coverage of services than is possible through the process monitoring described above. Periodic evaluations of these aspects of service delivery can provide useful information for reorienting and strengthening services. These periodic output-level evaluation exercises require special surveys and studies to be undertaken to obtain the necessary data.

Quality of STI Services
Methods for evaluating the quality of STI services have been tested and refined, and indicators form part of the recommended set of evaluation criteria for HIV prevention programs1, 2. STI service indicators 1 and 2 (previously called Prevention Indicators [PI] 6 and 7) look specifically at the curative and preventive aspects provided at health care facilities by STI case management providers. Key questions addressed in monitoring service quality include:

• Are services providing quality care for men and women infected with STIs?
• Are necessary drugs, diagnostics, and other commodities for treatment and prevention available?
• Are services used effectively as an entry point for HIV and STI prevention?

Protocols for measuring these indicators generally include enumerating health care facilities where STI case management is provided and evaluating the adequacy of health facility infrastructure for STI case management1. Other areas of quality that are assessed include provider attitudes toward STI clients, partner notification, and referral for voluntary HIV counseling and testing.

Several methodologies for evaluating the quality of STI services from providers have been compared3. The approach recommended by UNAIDS is a health facility survey with both provider interview and provider observation components. While the standardized methodology permits comparison across time and place, disadvantages include expense (observers may have to wait several days for a patient with STI symptoms to present) and the need to select sites with sufficient volume to permit observation. Additionally, biases related to the method of data collection itself can occur (for example, a provider may alter performance during observation and the knowledge elicited in interviews may not be reflected in actual patient management).Table 4-2 presents alternative strategies for assessing the quality of STI care, including use of mystery patients, exit interviews, and record reviews. Adaptations of these methodologies have also been used at other sources of care, including private practitioners, pharmacies, and "non-qualified" practitioners. Mystery patients (for health care facilities) or mystery shoppers (for pharmacies) are observers who are trained to pose as patients with an STI complaint. While this approach reduces bias that occurs when someone is aware of being observed, the expense may limit the number of observations or types of STI syndromes evaluated.

Reliable data on the quality of STI case management is important for effective management of STI control programs. The challenges of accurately assessing this indicator through health facility surveys should not be underestimated, however. In many cases, the choice of method is determined by what the health care providers will allow, particularly in the case of the private sector. For examples, private providers may participate in a short interview but will not allow their records to be reviewed or allow observations of their patient management sessions. Unless the physician or pharmacist community has consented, mystery client surveys may be viewed as intrusive and be ultimately counterproductive in standardizing high quality STI services in a community. It may be more practical to include a limited set of quality indicators into ongoing monitoring systems. For example, data routinely collected in patient registers, charts, or specially designed patient encounter forms can be analyzed to determine the proportion of patients managed according to protocol. A comprehensive approach would be to combine routine monitoring with periodic in-depth health facility surveys. Further discussion of these issues can be found in the review by Saidel and colleagues3. When this information is collected, data collection protocols and staff should strictly adhere to the principles of confidentiality and respect for persons.

In addition to monitoring the quality of services at the clinic level based on national guidelines, the continued applicability and appropriateness of STI case management guidelines should also be monitored periodically. Specifically, the etiologies of the various STI syndromes and the continuing susceptibility of Neisseria gonorrhoeae (and Hemophilus ducreyi if applicable) to the recommended antibiotics should be monitored. National guidelines may need to be altered and field tested to incorporate new diagnostic tests that may become widely available or to adjust based on shifting syndrome etiologies or changing antibiotic susceptibility of STI pathogens. Data collection efforts to determine the ongoing appropriateness of national STI treatment guidelines require special studies with high-quality laboratory support.

Coverage
As important as high-quality preventive and curative services are, their contribution to STI control is limited by the extent to which they are used, particularly by those at highest risk of acquiring and transmitting STIs. Evaluating coverage is therefore as important as evaluating quality. Several approaches to evaluating coverage are possible and the choice of methods will depend on factors such as marginalization of target groups, extent and nature of the informal health care sector, and the structure of service delivery. Mapping and sampling methodologies are used to quantify the number of risk settings and estimate the size of hard-to-reach populations (see Chapter 8, "Uses of Behavioral Data for Program Evaluation," Chapter 9, "Sampling Strategies for Monitoring HIV Risk Behaviors," and Larson et al.5 for more information on these methodologies). Where special screening and treatment services for sex workers are part of the public sector health system, these can be evaluated using adapted versions of the clinical evaluation tools described above. Where sex work is more covert and special outreach services have been developed, it may be more appropriate to evaluate those services using separate methods developed in the context of the project's objectives.

Whatever the internal approach to evaluating services, it should be complemented by an external view of the intervention's image and use. Community surveys should try to estimate the proportion of the target group that knows of and uses the services. Protocols and instruments designed for behavioral surveillance can include information about target groups' knowledge of health services. Special surveys and qualitative research among people with recent STI symptoms would provide information on health services use and access issues. Quantitative surveys looking at coverage should always be supplemented by qualitative research to clarify issues around barriers to care (see Chapter 12, "The Role of Qualitative Data in Evaluating HIV Programs," for a discussion of commonly used qualitative research methods).

Similarly, where identified bridge populations of men with occupational or other risks are included in interventions, evaluation should take into account both the quality of services offered as well as the extent to which those services are used. For example, in a city where migrant men are believed to be at risk, pharmacies may be a common source of care for treatment or prophylaxis. Evaluation of a program designed to improve such care should consider the quality of advice provided (through mystery shoppers or exit interviews), number and coverage of pharmacies, and service site knowledge and use though a survey of occupational risk groups supplemented by more in-depth qualitative research.

Effectiveness Evaluation: Outcomes and Impact
Evaluating the effectiveness of an STI control program involves estimating the impact of programs on the prevalence and incidence of STIs and STI-related behaviors, including treatment seeking behaviors, condom use and sexual behaviors (such as number and type of sexual partners).

To guide STI control efforts, a program manager would ideally monitor in the general population and in specific target groups the prevalence and incidence of each STI pathogen and related STI morbidity, as well as the prevalence of risk and treatment-seeking behaviors. In reality, this ideal is difficult to attain for a number of reasons:

• data from health care facilities often do not reflect what occurs in the community;
• population-based data on STIs are difficult and expensive to obtain;
• most commonly available diagnostic methods for STIs are inaccurate, particularly for women;
• accurate laboratory data on STI pathogens are expensive to collect; and
• reporting systems, where operational, rely primarily on reports from public sector health care facilities, while most STIs are managed at private or informal sector facilities that rarely report

Despite these obstacles, the outcome of STI control efforts on disease trends can often be estimated using a combination of passive data collection and special biologic and behavioral surveys, which together form a package known as STI surveillance. Table 4-3 summarizes some of the different components of STI surveillance. The definition and role of surveillance in STI control have been well established and refined over the years 6,7,8. Surveillance data in STI control programs are not only used to assess the effectiveness of programs but are used to determine the need for public health action. Additional critical program needs for which surveillance data are used include setting intervention priorities, planning and allocating resources, defining subgroups for targeted interventions, designing disease prevention programs, and conducting STI program process evaluation.

Passive Disease Surveillance
Most countries have some system in place for tracking numbers of STI diagnoses in public sector health care facilities, and reporting may include private practitioners as well. Despite major problems with completeness, reliability, and timeliness, passive STI surveillance has the advantage of being relatively inexpensive to implement. Common STI surveillance case reporting methods include:

• Health information systems (HIS)–Routine reporting of limited data, usually in the form of tally sheets, on a wide range of diseases seen at all public health care facilities
• Communicable disease reporting–Mandatory reporting of certain diseases of public health importance (presumptive or laboratory confirmed) from clinical and/or laboratory sites
• Enhanced surveillance from sentinel sites–More detailed data collection at a limited number of clinical and/or laboratory sites that are reinforced with training, regular supervision, and financial support

The choice of passive surveillance methods or combined passive and enhanced surveillance methods will depend on a number of factors, including the organization of STI service provision and availability and reliability of laboratory diagnosis. Data may be inaccurate because of underreporting and poor quality control of laboratory services, or unreliable because of lack of consistent case definitions.

In practice, STI surveillance markers may be limited by feasibility considerations to a few common, curable STIs or easily identifiable syndromes. Table 4-4 presents several syndromic indicators commonly used in passive STI surveillance.

In some situations, it may be useful to track etiologic markers of STI syndromes and include reporting from routine laboratory testing, such as antenatal syphilis screening. Cost and reliability of laboratory diagnosis are factors that may influence decisions about including laboratory surveillance.

Special Surveys and Studies
Where more reliable estimates of STI prevalence are needed and a supplement to the passive reporting system is desired, periodic population-based STI prevalence and behavioral surveys can be carried out on the general population or specific target groups. Although considerably more expensive than passive surveillance, such surveys can provide accurate estimates of disease prevalence in specific populations. Availability of highly accurate urine-based diagnostic tests for several common STIs makes population-based surveys feasible. Because of the high cost of such tests, surveys are most practical for monitoring trends among high-risk priority populations where initial prevalence is high (and sample sizes smaller). When combined with behavioral surveillance, STI prevalence surveys can provide valuable information about disease and risk behavior trends in priority populations, where most sexual transmission occurs.

These special STI surveillance activities, although complex and currently expensive, provide extremely valuable data for evaluating the effectiveness of STI and HIV prevention programs. The recommendations for Second Generation HIV Surveillance, which advise the collection of both biologic and behavioral data, include STI surveillance methods (syndromic reporting, syphilis seroprevalence, STI prevalence surveys among high-risk groups) as a complement to other HIV and behavioral surveillance activities2. The role of behavioral surveillance is discussed in more detail in Chapters 8-13 of this Handbook. Other issues of importance for STI control as an HIV prevention intervention, such as modeling the impact of STI interventions on HIV transmission and measuring the efficiency and cost-effectiveness of different STI control interventions, are discussed in Chapters 14-17.

Other surveillance activities that fall in the category of special studies include assessments of incidence and prevalence of STI-related complications (such as pelvic inflammatory disease [PID] or cervical cancer), prevalence surveys of viral STIs (such as herpes simplex virus [HSV] type 2 or hepatitis), and estimation of the economic costs of STIs, to name a few. This information helps to refine and refocus mature STI control programs but are less essential when initial efforts should be directed at maximizing effective treatment opportunities for the bacterial STIs.

Controlled clinical trials, either community randomized or individual randomized, in assessing the impact of STI control on HIV incidence9 or treatment trials assessing the impact of chronic suppressive HSV therapy are difficult and expensive and not warranted in routine monitoring and evaluation of programs.

Other Considerations

Laboratory Issues
The laboratory is an essential component of STI control programs and in evaluating STI control program interventions. The availability of quality laboratory services are severely limited in most settings due to constraints of cost (both initial and recurrent), expertise, inconsistencies in supplies, and erratic quality control measures and other support. Additionally, laboratory tests for specific STI pathogens vary widely in sensitivity and specificity to detect specific etiologic agents and each test has specific requirements in terms of specimen handling and processing10. As such, interpretation of laboratory-based STI trends must consider issues of test quality. For example, sudden declines in syphilis prevalence in antenatal clinic populations may be due to outdated reagents and lack of quality control procedures in place rather than an effective program. Similarly, a very high reported prevalence of chlamydial infection may be due to inexperienced technicians who over-interpret fluorescent slides rather than to a severe chlamydial infection problem.

The availability of urine-based DNA amplification testing has now made population-based STI surveys feasible and allows for surveys to be conducted at great distances from laboratory support. When conducting these surveys in the field, special attention needs to be paid to specimen handling to ensure quality laboratory data. Finally, changing test methodologies can sometimes have a dramatic effect on measured prevalence rates of STI pathogens. As such, the type and the accuracy of diagnostic tests must also be taken into account when interpreting STI trends or comparing STI prevalence rates among different sites.

STIs as Biologic Proxies for HIV Transmission
The relationship between STIs and HIV is complex because they act both as confounders and effect modifiers (Figure 4-3). STIs facilitate HIV transmission through numerous biologic mechanisms that affect both infectiousness and susceptibility. HIV infection alters the natural history of some STIs. Moreover, the predominant mode of HIV transmission globally is through sexual contact; sexual behavior, partner type, and sexual network can affect both STI and HIV risk.

STI prevalence and incidence, both increasing and decreasing, are being promoted as a proxy measure for the effectiveness of HIV prevention programs because STIs, like HIV, are markers of unprotected sex. New cases of bacterial STIs, which are curable, ought to reflect high-risk sexual activity1. An STI with high infectivity, short incubation period and period of infectiousness, and little resulting immunity would be a sensitive marker for unprotected sexual contact. Conversely, HIV prevention interventions should reduce STI prevalence and incidence rates more rapidly than the prevalence and incidence of HIV infection and, as such, bacterial STI measures would be a useful indicator of recent changes in sexual behavior for HIV1.

This commonsense notion of STIs being a marker for the effectiveness of HIV prevention programs actually belies several complex interrelationships and processes11. First, STI is a term that refers to a diverse group of about 30 pathogens–bacterial, protozoal and viral, each with unique biologic features of infectiousness, host response, and associated patterns of sexual contact. Second, STI transmission dynamics in a population are significantly influenced by subsets of the population, the so-called core and bridge populations. As such, changes in the incidence of bacterial STIs, such as gonorrhea, in a population might be due primarily to behavior changes, such as increased barrier method use, of these population subsets. It is conceivable that a decline in gonorrhea may occur in a population where HIV transmission would continue to take place, because people infected with HIV remain infectious for years. Third, effective treatment will have a dramatic impact on the incidence of treatable STIs. Consequently, one cannot attribute changes in STI incidence solely to changes in behavior. Finally, STI changes must be put in the context of longer-term trends, where such data are available.

While STI prevalence and incidence rates are important to monitor for STI and HIV control program purposes, the STI data need to be triangulated with behavioral information to fully understand the effect of HIV prevention efforts.

Renton and Whitaker distinguish two types of STI indicators, indirect and direct, for HIV control programs11. An indirect indicator would be defined as an STI whose occurrence is used as a proxy measure of high-risk sexual activity. They propose that the ideal characteristics of such an STI would include high infectivity, short period of infectiousness, short incubation period, and absence of conferred host immunity. Neisseria gonorrhoeae might be an appropriate STI for an indirect indicator. On the other hand, a direct indicator would be an STI whose occurrence is used as a proxy measure of HIV infection itself. Characteristics of such an STI would include life-long infection and life-long infectiousness. Practically, a direct indicator of HIV might be used to assess the potential for HIV spread in populations or as a tool to monitor HIV programs without measuring HIV infection directly. Such a direct indicator has yet to be defined.

As STI, behavioral, and HIV data are collected on populations an understanding of their interrelationship will increase. This increased understanding may help to improve surveillance systems and make interpretation of existing surveillance data stronger.

Conclusion

STI control programs must adapt strategies that respond to the heterogeneity of STI pathogens and transmission dynamics within their populations as well as the limitations of finance, infrastructure, and personnel. As such, STI control programs vary widely among countries and among sub-populations and, consequently, monitoring and evaluation of such programs will vary. There are, however, essential elements of monitoring and evaluation regardless of the program type:

• Formative evaluation–As a first step in the program cycle, information should be collected to describe the current situation, identify need, and design an appropriate response

• Process evaluation–These data monitor how efficiently program activities are carried out and help keep services on track. Much of the data for process evaluation can be collected routinely as part of service delivery. Monitoring the quality and coverage of STI services is essential and requires special studies and surveys to be undertaken

• Effectiveness evaluation–These data estimate the impact of the STI control program on the prevalence and incidence of STIs and STI-related behavior both in the general population and in specific target populations. Despite obstacles, outcomes of STI control efforts on disease trends can be estimated using a combination of passive data collection and special biologic and behavioral surveys–an STI surveillance system

The STI laboratory is an important component of STI control programs and in the evaluation of STI control program activities. The availability of laboratory services is severely limited in most settings due to constraints of money, technical expertise, commodity supply and quality control. As such, the cost of high-quality testing must be taken into account when planning laboratory-based evaluations, and the type and sensitivity of diagnostic tests must be considered when interpreting STI trends or comparing STI prevalence rates among different sites or populations. STI trends as biologic markers for HIV prevention programs are best interpreted with behavioral information in the population because treatment and specific interventions in sub-populations can affect STI prevalence in the general population.

References

1. UNAIDS/00.17E. National AIDS programmes: a guide to monitoring and evaluation. Geneva: Joint United Nations Programme on HIV/AIDS (UNAIDS); 2000.
2. WHO/CDS/CSR/EDC/2000.5 and UNAIDS/00.03E. Guidelines for second generation HIV surveillance. Geneva: World Health Organization and Joint United Nations Programme on HIVAIDS; 2000.
3. Saidel T, Vuylsteke B, Steen R, et al. Indicators and the measurement of STD case management in developing countries. AIDS 1998;12(Suppl 2):S57-S65.
4. Schwartländer B, Van Den Hoek A, Heyman D, Gerbase A. Surveillance. In: Dallabetta G, Laga M, Lamptey P, editors. Control of sexually transmitted diseases. Arlington (VA): Family Health International; 1998.
5. Larson A, Stevens A, Wardlaw G. Indirect estimates of "hidden" populations: capture-recapture methods to estimate the numbers of heroin users in the Australian capital territory. Soc Sci Med 1994;39(6):823-831.
6. WHO/CHS/HIS/99.2. Guidelines for sexually transmitted infections surveillance. Geneva: Joint United Nations Programme on HIV/AIDS and World Health Organization; 1999.
7. Ward JW, Greenspan JR. Public health surveillance for HIV/AIDS and other STDs: guideposts for prevention and care. In: Holmes K, Marling P, Sparling P, et al., editors. Sexually transmitted diseases. 3rd ed. New York: McGraw-Hill Inc.; 1999. p. 887-892.
8. Catchpole MA. The role of epidemiology and surveillance systems in the control of sexually transmitted diseases. Genitourin Med 1996;72(5):321-329.
9. Grosskurth H, Gray R, Hayes R, et al. Control of sexually transmitted diseases for HIV-1 prevention: understanding the implications of the Mwanza and Rakai trails. Lancet 2000;355(9219):1981-1987.
10. Van Dyck E, Meheus AZ, Piot P. Laboratory diagnosis of sexually transmitted diseases. Geneva: World Health Organization; 1999.
11. Renton AM, Whitaker L. Using the classical STDs to monitor HIV prevention programmes. In: Paccaud F, Fader JP, Gutzwiller F, editors. Assessing AIDS prevention. Basel (Switzerland): Birkhauser Verlag; 1992. p. 32-51.