Use of ‘bogus distractors' in questionnaire design: a methodological analysis

Objectives: The use of ‘bogus distractors', one long-respected principle of questionnaire design, rests on the concern that respondents ‘seek to please the interviewer' and ‘seek to look knowledgeable' by answering ‘yes' to ‘have-you-heard-of' type questions. The primary goal of this analysis was to determine whether the use of ‘bogus distractors' still serves to distinguish respondents with a tendency to be ‘yes-sayers' from those who are ‘straight-shooters', given the widely recognized societal decrease in respect for authority over the past 50 years.

Methods: In order to distinguish ‘yes-sayers' from ‘straight-shooters' in a line of inquiry to determine if subjects had ever heard of the Tuskegee Syphilis Study, the key question-of-interest was inserted into a series of 8 ‘have-you-heard-of' questions, naming 4 ‘bogus studies' and 4 ‘real studies'. The individuals who responded ‘yes' to having heard of 2 or more of the bogus questions were identified as “yes-sayers”, all others were categorized as ‘straight-shooters'. All questions were administered in the Tuskegee Legacy Project (TLP) Questionnaire in a 1999-2000 NIH-funded Random Digit Dial (RDD) survey of 1,133 adults in four U.S. cities.

Results: Overall, for the 1,133 respondents, 31.6% were identified as ‘yes-sayers'. Blacks had the highest rate percentage of ‘yes-sayers' (42.8% vs 35.7% for Hispanics and 24.2% for Whites, p <0.05). Percentage of ‘yes-sayers' differed by income (37.5%, 31.0% and 24.4% for <$0,000, $20-74,999, and >$75,000, p < 0.05) with parallel findings for education levels, but not by sex, age groups, or cities.

Conclusion: These observed significant differences in percentage of ‘yes-sayers' across several key demographic variables suggests the need for further research to determine whether these levels of observed differences in ‘yes-sayers' has a measurable impact on substantive findings as reported from survey research.

This study was supported by NIDCR/NIH grant U54 DE 14257 and T32 DE 007255.