The onset of the HIV epidemic has made the validity and reliability of measurements of sexual behaviors, particularly behavioral change and condom use, critical and salient. The availability of sexual behavioral data is vital to developing and monitoring appropriate prevention programs, but their effectiveness depends to a large extent on their quality. However, validating sexual behavior data–in other words, assessing the degree to which reported behavior reflects actual behavior–is not easy. The most obvious difficulty is that direct observation or public records are virtually impossible in the context of private sexual behavior. By implication, sexual behavior data obtained only through questions represents reported, not observed behavior. Hence, there is a need for caution in using these data for policy purposes or as the basis of education programs1,2.

For decades, questions on sexual activity have been avoided in most health or fertility surveys on the grounds that such questions were morally and culturally sensitive and the responses were of low reliability and validity. However, many of the problems raised about validity are not specific to sexual behaviors. Questions about child death, fertility, marital status, and income are also highly sensitive. The extent to which individuals are willing to recall and report details of past sexual behavior may indeed greatly vary according to gender, age, social status, risk behavior and many other cultural conditions. In some communities, certain behaviors may be actively concealed because they are illegal or considered socially unacceptable, such as prostitution, homo- or bisexuality, adultery, and premarital sex. In some cultures, men may overestimate their number of sexual partners, and in most societies, women are more likely than men to underestimate their sexual activity outside marriage.

Validity is improved when the research variables to be incorporated in the questionnaire or in the interview guide are tested during in-depth interviews in which the meaning of the language used and the accuracy of recall are checked. Accuracy of recall is an important issue because it can be influenced in a number of ways and it has been shown to vary greatly according to the emotional significance of the event. For example, a person is more likely to accurately recall an emotionally important event, such as age at first sex, than to recall details of regular visits to sex workers. Memory error is more likely to occur for frequent events than for single ones. Respondents with multiple partners may better remember their number if asked to give detailed characteristics of each partner, starting with the most recent one. The reference period should also have meaning for respondents, such as a month, a year or lifetime, rather than artificial periods, such as 3 or 6 months. In addition, a very short reference period may maximize the accuracy of responses but may not capture infrequent behaviors in a particular population.

Pilot testing of the questionnaire should also facilitate decisions about its sequence and structure. It has been shown that validity is improved when the questionnaire is structured in such a way that sensitive questions are asked only to appropriate respondents by using filters and skips, and are placed near the end of the interview.

Another issue of sexual behavior measurement error is the social desirability bias, sometimes called self-presentation bias, that can partly be addressed by careful wording of the questions, by a non-judgmental attitude of the interviewer, and by ensuring maximum privacy. However, it is also recognized that the social pressure to give desired responses may vary greatly according to the perception of the appropriate preventive behaviors. "Increased" condom use after intense prevention programs is one example.

Several cross-national standard questionnaires on sexual behavior have been developed in the past 10 years and used in various cultures that have different sensitivities to sexual activity3,4. Core questions about first sex, number of sexual partners, and condom use yield answers that seem to be quite comparable across and within cultures. Other topics, such as commercial sexual contact and risk perception, seem much more ambiguous.

Improving Participation Rates

Drawing a probability sample of potential respondents–household- or non-household-based–for surveys of sexual behavior was long considered an impossible task. However, recent experience has demonstrated that it is feasible and that participation rates in sexual behavior surveys are as high as in other surveys when several precautions are taken. These precautions also serve to increase the legitimacy of the survey. They are:

• informing the community and the selected individuals or households in advance with letters, radio messages, or community leader visits, and explaining the purpose of the survey;
• selecting interviewers in terms of age, marital status, and gender in order to better match cultural norms on sexual communication;
• training interviewers before going to the field with emphasis on rapport building, talking about sexuality, and overcoming embarrassment;
• including a provision in the survey for a large number of call-backs;
• guaranteeing anonymity and confidentiality to the respondent; and
• ensuring privacy during the interview.

In many developing countries where survey teams have used careful procedures, response rates close to 80-90 percent have been achieved5. These rates have been less in urban areas than in rural areas and less in high income areas than in lower. Certainly, although these high response rates are reassuring, they do not guarantee validity if participation is highly selective in terms of risk behaviors.

Methods For Checking Validity And Reliability

Several methodologies can be implemented at a minimal cost and contribute to establishing validity and reliability.

Consistency with Independent Sources
One of the objectives of most sexual behavioral surveys is to interview a representative sample of men and women. A comparison of estimates of socio-demographic characteristics derived from these surveys (for example, age/sex composition, marital status, urban/rural residence, educational level, occupation) with similar estimates obtained from independent sources of information may provide insights into their representativeness and the extent to which these findings can be extrapolated.

Observed deviations may be due to measurement error and/or sample bias. Non-response rates, including refusal rates and participation bias, are a major concern because of the stigma attached to AIDS. This non-response may simply vary according to repeated absence of families or individuals. In some instances, absence means that selected respondents refuse to participate in the survey. There is limited documented evidence that individuals with high-risk behaviors may be less likely to participate in surveys that address sexual health. However, if HIV/AIDS is explicitly the focus of the questioning, fear and denial may lead individuals with high-risk behaviors to participate less6.

Usually, non-response rates are not different for sexual behavior surveys than for those of other types of surveys using similar methods of sampling. However, because non-respondents in developing countries are often urban male youth and mobile people, their lack of participation may affect the validity of survey results that focus on risk behaviors. Non-household-based surveys, such as school- or workplace-based surveys, that target specific sub-population groups may lead to increased participation if privacy and confidentiality are guaranteed, although the effects of place of interview on validity is still unknown.

When comparing socio-demographic indicators such age/sex composition, marital status, urban/rural residence, or educational level with other data sets, other demographic indicators, such as mean or median age at first marriage, may be useful to consider. Median age at first sex and median coital frequency may be added if other data sources exist and if definitions of marriage and first sex are the same in both surveys. The same principles may, in theory, be applied to assess the aggregate validity of sexual behavior parameters, such as sexual contacts before or outside marriage, reported symptoms of sexually transmitted infections (STIs), or condom use within different partnerships.

However, the possibilities of comparison are usually quite limited because of the absence of independent sources or because of differences in timing of the survey or sampling methods. In a few instances, some data are available in family planning clinics, STI clinics, or health centers, but individuals included in household surveys are likely to be very different from those who report to clinics. Data on condom sales and distribution may be of use, at least as a plausibility check. Commercial sex workers' reports on condom use may be compared with those of clients7.

Internal Consistency Checks
A number of internal consistency checks may elucidate the variation of the quality of response over the recall period. These checks depend directly on the type of question asked in particular questionnaires. According to the WHO/GPA's definition of core prevention indicators, the key variables to assess should include: age at first sex; sex with a non-regular partner in the last 12 months; condom use in the last sexual intercourse with a non-regular partner; reported symptoms of STIs; and knowledge of ways to prevent HIV transmission. Comparisons may thus include: direct versus indirect measure of these specific variables; aggregate reporting of women versus men; aggregate reporting of wives versus husbands (when possibilities exist of linking husband and wife responses). A limited number of aggregate male/female comparisons of coital frequencies and condom use have been made with data sets from five countries where the WHO/GPA protocol was used8. Generally, the correspondence was reasonably high, particularly over short time periods.

Tests of plausibility should also be conducted when associations between variables are known or when different questions on the same topic are asked. For instance, coital frequency between partners usually decreases with the duration of marriage/partnership; age at first sex after age at first marriage should be uncommon; ever use of condoms versus use of condoms as a contraceptive method versus use of condoms as behavioral change due to AIDS may be compared.

Convergent Validity Checks
Convergent validity is usually used to describe the level of agreement between results provided from different methods of data collection (triangulation), such as face-to-face standard questionnaires, diaries, in-depth interviews, self-completion of questionnaires, or self-completion of sensitive questions (limited to literate populations). Ideally, the comparison should be performed by repeat measures on the same sub-sample and in a short period of time.

However, in the context of sexual behavior, interpreting differences between different methods is not straightforward because of uncertainty about their relative validity. A high level of consistency between two approaches is reassuring; conversely, inconsistencies should stimulate further research on specific variables. The pattern of inconsistencies might also suggest which method yields more valid information9.

If differences in results are found between structured face-to-face questionnaires and in-depth interviews on the same variables among the same sample population, they may be explained by a variety of factors: the quality and training of interviewers; the time spent to establish rapport and trust-building between interviewer and respondent; the time spent for recall and reflection; the explanation of the meaning of the questions; the flexibility and adaptability of the questioning.

Comparison of daily diaries with retrospective reporting from interviews may provide a useful way of checking recall accuracy. However, willingness to keep sexual diaries might introduce a problem of selection bias10.

Qualitative methods usually require the "play-back" method, which means the continual interchange of ideas between investigators and informants and the verification of research hypotheses.

Techniques to Assess Reliability
Reliability usually refers to the ability of a method to give consistent results over many tests, repeated at different times. While low reliability always means low validity, high consistency of responses across several measurement may reflect a constant bias. Reliability may be directly assessed by test-retest measures, using the same instrument and method. It can also be assessed indirectly by examining internal consistency. Dare and Cleland recently reviewed five reliability studies conducted in various countries with retest subsamples of about 300 respondents at an interval of 2 to 6 weeks after the main survey8. Consistency varied not only according to specific variables and questions but also according to the characteristics of the study population. At the group level, however, reliability was acceptable for most of the key sexual risk behavior factors.

Biomedical Validation
Validating self-reports with biological or clinical markers of STIs is a research strategy that has been used in some studies4. Community surveys with appropriate probability sampling procedure and serious ethical precautions may include sampling of blood, urine, or saliva specimens for HIV and STI testing or clinical examination. Such surveys have achieved high rates of participation. Anonymity of the respondent is guaranteed but special procedures may allow researchers to link individual reporting on risk behavior to biological markers at the data processing stage. However, the major limitation is that the association between sexual partnerships, number of sexual partners, and STIs is far from straightforward. Only a limited number of validity checks may be made using this method, such as consistent condom use versus recent STIs or no sexual activity and the presence of STIs, and even these may be confounded by other factors.

Conclusion

Small- or large-scale surveys have an important role to play in monitoring behavioral change over time or to evaluate program impact. However, despite a growing body of knowledge accumulated on how to conduct such surveys, validity measures are still to a great extent lacking. Obviously, data quality from sexual behavior surveys critically depends on high standards of execution at each phase of the survey: planning, implementation, analysis, and interpretation. The issue of representativeness of the study population should also be a major concern. Whether the study sample is randomly selected or of convenience, a key factor is to ensure that all reasonable efforts are made to achieve the highest level of response or representativeness, and that the effects of selection bias are documented and integrated into the analysis. This will be critically important to keep consistency between successive surveys.

Indeed, the same type of instrument, interviewer, training, fieldwork strategy, and supervision should be used in successive surveys. This requires a detailed documentation of the baseline survey, including definition of the population of interest and biases in participation. In AIDS surveys, some of the individual characteristics of most concern to the analysis tend to be those associated with unconventional lifestyles and, thus, respondents may be more difficult to reach for interview. This bias cannot be prevented by interviewing more accessible substitutes. Rather, interviewers must strive to interview every selected person. Training interviewers in nonjudgmental attitudes, careful question structure, and culturally appropriate wording are important strategies to reduce respondent self-presentation bias.

Surveys on STIs, sexual behavior, and HIV require special attention to issues of informed consent, anonymity, and confidentiality. The effect of the location of the interview (household or other settings such as clinics, workplaces, or community centers) on respondent's perception of privacy has not yet been adequately measured.

The lessons learned from previous validity and reliability checks in developing countries show that evidence is varied and that the acceptability of measurement errors may well depend on the objectives of the surveys. For example, a constant bias over time is less important for monitoring trends than for developing an appropriate intervention or making epidemiologic forecasts. Validity checks of different kinds are needed that are compatible with time and resources pressures in order to identify errors and biases and to be certain that behavioral changes are real and not due to the unreliability of reports. Triangulation is especially needed not only to validate the accuracy of responses but also to provide a context for the interpretation of quantitative indicators.

References

1. Miller HG, Turner GF, Moses LE, editors. Committee on AIDS Research and the Behavioral, Social, and Statistical Sciences, National Research Council. AIDS, the second decade. Washington (DC): National Academy Press; 1990. Appendix C, Methodological issues in AIDS surveys; p. 206-241.
2. Caraël M, Cleland J, Deheneffe JC, et al. Sexual behavior in developing countries: implications for HIV control. AIDS 1995(10); 9:1171-1175.
3. Caraël M. Sexual behavior. In: Cleland J, Ferry B, editors. Sexual behavior and AIDS in the developing world. London: Taylor & Francis; 1995.
4. Caraël M, Cleland J, Ingham R. Extra-marital sex: implications of survey results for STD/HIV transmission. In: AIDS prevention and impact in the developing world: demographic and social science perspectives. Health Transit Rev 1995;4(Suppl):153-173.
5. Ferry B, Deheneffe JC, Mamdani M, Ingham R. Characteristics of surveys and data quality. In: Cleland J, Ferry B, editors. Sexual behavior and AIDS in the developing world. London: Taylor & Francis; 1995.
6. Konings E, Bantebya G. Validation of the population survey protocol for the measurement of HIV/STD prevention indicators in Kampala, Uganda. Geneva: World Health Organization/Global Programme on AIDS; 1993.
7. Pickering H, Todd J, Dunn D, Pepin J, Wilkin A. Prostitutes and their clients: a Gambian survey. Soc Sci Med 1992;34(1):75-88.
8. Dare OO, Cleland JG. Reliability and validity of survey data on sexual behavior. In: AIDS impact and prevention in the developing world: demographic and social science perspectives. Health Transit Rev 1994;4(Suppl):93-110.
9. Konings E, Bantebya G, Caraël M, et al. Validating population surveys for the measurement of HIV/STD prevention indicators. AIDS 1995;9(4):375-382.
10. Hogsborg M, Aaby P. Sexual relations, condom use and perceptions of AIDS in a urban area of Guinea-Bissau with a high prevalence of HIV-2. In: Dyson T, editor. Sexual behavior and networking: anthropological and socio-cultural studies on the transmission of HIV. Liege: Ordina; 1992.