Visit fhi.org in: Español | Français | Russian | Arabic
 Search fhi.org:
 

HIV/AIDS

Evaluating Program Effectiveness

Email this to a friend

Find related documents

Evaluating the effectiveness of HIV/AIDS programs requires collecting data to measure the extent to which program objectives are achieved. Outcome and impact assessments almost always require quantitative measurements, which will help identify how well the program objectives were achieved. Outcome and impact evaluations can explain:

  • What outcomes were observed?
  • What do the outcomes mean?
  • Does the program make a difference?

Program evaluation is a complex undertaking because we lack a complete understanding of how different behaviors and epidemiological factors influence evolving epidemics. While changes in HIV prevalence might reflect the long-term impact of multiple interventions, it is difficult to "prove" that decreasing prevalence resulted from prevention work. Other factors -- mortality, migration or epidemic saturation of the population at risk -- also might explain such changes. Given these limitations, a comprehensive approach to effectiveness evaluation relies on multiple techniques to examine the relationship between biomedical, behavioral and socio-demographic data. It emphasizes links between outcome data from program interventions, patterns of HIV prevalence, and estimates of cost-effectiveness. This broad, integrated strategy has proven beneficial for many reasons:

  • It assesses program effects from different perspectives.
  • It produces thorough, reliable information from three sources.
  • By offering a combined analysis of various data sets, it provides context for interpreting and explaining observed epidemiological HIV/STI trends.
  • It can generate crucial data over time, since it can be performed at different periods.
  • It enhances the use of behavioral surveillance data by analyzing them in conjunction with process evaluation data and results from qualitative research.

Incorporating evaluation at the program design stage is essential to ensure that research activities produce useful results. Planning an intervention and designing an evaluation strategy are inseparable activities. To ensure the relevance, sustainability and lack of duplication of evaluation activities, project designers must collaborate with local stakeholders (e.g., National AIDS Control Program, Ministry of Health, other donors) to outline the process. An evaluation plan should contain:

  • Scope of the evaluation
    • goals and objectives
    • conceptual framework correlating inputs, processes, outputs and outcomes
    • ethical issues
  • Methodological approach
    • study design
    • indicators
    • means of verification
    • interval between data collection points
  • Implementation plan
    • selection of geographic areas
    • roles and responsibilities
    • timetable for identified activities
    • preparation of budget
  • Dissemination plan for evaluation results

The monitoring and evaluation pipeline below illustrates that few projects ultimately warrant an evaluation of the effectiveness of their implemented prevention activities. Individual projects that implement standard intervention strategies that have already been found effective in similar settings should focus their evaluation activities on formative evaluation (when needed for project planning), process monitoring and capacity-building assessment. Only in the case of a demonstration project would there be justification for a more rigorous research design at the project level. Effectiveness evaluation should be at the program level, not the project level.

Monitoring and Evaluation Pipeline Diagram

Resources

  1. AVERT: A Tool for Estimating Intervention Effects on the Reduction of HIV Transmission. Family Health International, 1999.
  2. Evaluating Programs for HIV/AIDS Prevention and Care in Developing Countries: A Handbook for Program Managers and Decision Makers. Family Health International, Forthcoming in September 2001.
  3. Meeting the Behavioural Data Collection Needs of National HIV/AIDS/STD Programmes. Proceedings from a joint IMPACT/FHI/UNAIDS workshop. Arlington and Geneva, 1998. Family Health International/IMPACT and UNAIDS: Available on website
  4. Rossi R, Freeman H. Evaluation: A Systematic Approach. 4th Edition. Sage Publications, Newbury Park, California. 1989.
  5. UNAIDS/WHO, Working Group on Global HIV/AIDS and STI Surveillance. Second Generation for HIV. Compilation of basic materials (CD ROM available from UNAIDS/WHO). January 2001.
  6. WHO/CDS/CSR/EDC/2000.5, UNAIDS/00.03E. Guidelines for Second Generation HIV Surveillance. Geneva: UNAIDS/WHO 2000. Available on website www.unaids.org
  7. WHO/GPA/DIR/88.8 Sentinel Surveillance for HIV infection: A method to monitor HIV infection trends in population groups. Geneva: World Health Organization, 1988.
  8. UNAIDS/00.17E National AIDS programmes: a guide to monitoring and evaluation. Geneva: UNAIDS, 2000.