Changes in self-reported behaviour amongst a cohort or sample

Much of the evidence regarding the effectiveness of HIV-prevention measures comes from self-reported risk behaviour, which may provide information on:

• self-reported condom use
• self-reported instances of unprotected sex
• self-reported instances of serodiscordant unprotected sex.

These measures are more commonly used in prevention studies than clinical outcome measures, with the result that meta-analyses of studies usually have to use these to compare effectiveness.

In randomised controlled trials or prospective cohort studies, groups of people are enrolled at the beginning of an intervention, or at the start of the study period, and followed through the study period to assess changes in behaviour. Studies are reliant on the self–reported sexual or drug–using behaviour of participants, which may be unreliable. People’s reports of their behaviour may be distorted by ‘recall bias’ (being unable to remember their behaviour accurately) and by ‘social desirability bias’ (a tendency to understate or forget about stigmatised behaviour and overstate valued behaviour). 

A range of techniques have been developed by behavioural researchers to reduce these potential biases.

For example, Project Sigma, a UK investigation of gay men's sexual behaviour, used two methods to elicit information about reported sexual behaviour: one was the standard questionnaire method; the other was the process of keeping a sex diary over the period of a month. Significant discrepancies in self-reporting were noted when the two accounts were compared by researchers.¹

Computer-assisted self-interview (CASI) is a technique that has been used in a range of settings. Rather than be interviewed by a researcher, study participants complete a survey online or on a computer in the study centre - this has been shown in a number of studies to improve accuracy of reporting. A version of CASI called Audio-assisted Computer Self Interview (ACASI) has been developed for people less comfortable with filling in computer forms. Here, the subject gives spoken answers via a headset to questions either read on a form or read out by a computer-generated voice, while sitting in a private room.

In one RCT² of ACASI versus face-to-face interviewing in 139 female and 259 male sex workers in Mombasa, Kenya, for instance, ACASI yielded considerably higher figures for a number of different risk behaviours. However 20% of the original sample had to be excluded because they could not read the questions, which were presented on screen.

ACASI captured a significantly higher median number of regular partners (two versus one, both genders) and casual partners (three versus two in women, two versus one in men) in the last week. It uncovered much higher figures for certain behaviours and experiences regarded as taboo or shameful: group sex amongst men (21.6 versus 13.5%), intravenous drug use (10.8 versus 2.3% in men; 4.4% versus zero in women) and rape of males (8.9 versus 3.9%). A surprisingly high number of women reported in ACASI that they had paid for sex (49.3 versus 5.8%).

In contrast, behaviours that had already been intensively researched because they formed the enrolment criteria for the study (anal sex, sex work, sex between males) were actually reported less frequently in ACASI.

Researchers have been aware of the problems of recall bias, social-desirability bias and so on for decades, but findings from some biomedical-prevention studies have suggested that trial subjects may misreport risk behaviours or adherence to an even greater degree than previously thought. To give some recent examples, there have been large discrepancies between self-reported adherence to the trial protocol and actual adherence as measured in a microbicide trial³ and a trial of pre-exposure prophylaxis (PrEP).⁴ In the latter, actual adherence, as measured by biomedical methods, was 43 to 51% while adherence on the basis of self-report was claimed to be 94 to 95%.

A South African study presented at the 2010 Microbicides Conference⁵ found that condom use may be similarly over-reported by some groups and may, in some cases, be less than half of the use reported. Gafos and colleagues found that this was mainly due to women reporting consistent use when, in fact, their use was inconsistent. If condom use and other protective behaviours are, in fact, over-reported in trials, due to social-desirability bias, this could go a long way towards explaining why, in a number of trials, reports of behaviour change have not been accompanied by similar changes in HIV or STI infections.

Another example in which social-desirability bias may considerably distort the real situation is the vexed question of how common homosexual desire and behaviour is. In September 2010, for instance, the UK Office for National Statistics (ONS) issued a report⁶ in which, based on phone interviews with a random sample of 238,206 adults over 16, they only found 1.5% of respondents identifying as lesbian, gay or bisexual and 0.5% as ‘other’, leading to the estimate that there were 726,000 to 968,000 people in the UK who identified as gay, bisexual or at least non-heterosexual.

The gay-dating website Gaydar issued a press release⁷ pointing out that 2,185,072 men and women in the UK were at that point registered on Gaydar alone, equating to 6.7% of the UK population. While some of these could be duplicate profiles or non-nationals, this suggested that the headline figure ONS presented (1.5% lesbian or gay) was a gross underestimate – even though the ONS quoted household surveys that came to similar figures.

One thing the ONS did not mention in its headline estimate of 1.5% for the proportion of the population who were gay/bisexual was that another 3.3% of respondents either refused to answer sexual-orientation questions or simply did not respond to them. If the majority of these were doing so because of homosexual or at least non-heterosexual orientation, then that would bring the total up to near the 5% often quoted as an estimate for the proportion of the population who are gay. Another factor is that gay people are more likely to live in single-person households and that the telephone-interviewing process may therefore under-sample them.