Selection Bias in Online Polls: Twitter vs. Cognitive Daily
There was an uproarious SmackDown on Twitter between @brucewagner and @davemunger over the issue of polling the percentage of gay/homosexual tweeters/blog readers.
You see, there's this small issue called selection bias that prevents reputable pollsters from asking a question in this way: How gay is Twitter? ;) Are you gay?
selection biasSelection bias comes in two flavors: (1) self-selection of individuals to participate in an activity or survey, or as a subject in an experimental study; (2) selection of samples or studies by researchers to support a particular hypothesis.Was Bruce actually serious that his Are you gay? poll was completely unbiased?
Yes, that's right, only 54% of all Twitterers self-identified as straight in this fabulous poll, with 28% calling themselves gay and the rest in between. Off just a little?
Sexual Behavior and Selected Health Measures: Men and Women 15-44 Years of Age, United States, 2002Sexual orientation
In response to a question that asked, “Do you think of yourself as heterosexual, homosexual, bisexual, or something else?” 90 percent of men 18-44 years of age responded that they think of themselves as heterosexual, 2.3 percent of men answered homosexual, 1.8 percent bisexual, 3.9 percent “something else,” and 1.8 percent did not answer the question. Percents for women were similar.
[See Alkaline Earth for more.]
Given the wildly inaccurate Twitter numbers, Dave Munger wanted to prove a point by posting his own Are you homosexual? poll to the readers of Cognitive Daily.
He then compared these results to an earlier "Casual Friday" poll, which embedded a query about same- vs. opposite-sex partners in a series of questions about romantic gifts:
Selection bias and homosexualityHowever, some commenters noted that the Romantic Gift poll was not free of bias, either. Heterosexual partnered individuals were perhaps more likely to answer questions on whether jewelry, perfume, chocolate, flowers, lingerie, etc. constituted romantic gifts or not.A couple hours ago I posted a quick poll, in what might be construed as an unbiased fashion. I simply asked respondents for their sexual orientation, offering a wide array of choices ranging from "straight" to "mostly gay" to "gay" to "other."
In fact, my poll was biased -- not because the question itself was slanted, but because of the way respondents were recruited: I titled the post "Are you homosexual?" Potential respondents who are homosexual or who don't have traditional sexual preferences are more likely to be interested in the question, and therefore more likely to respond. How do I know this biased the sample? Because I collected similar data last week in the Casual Fridays survey about romantic gifts. In that survey, women reported same-gender partners 5.7 percent of the time, and men reported same-gender partners 3.7 percent of the time.
The moral of the story?
Do not give sexy nightwear or sex toys to women on the first date...
- Which Is Worse: Your Partner Having A Heterosexual Or Homosexual Affair?
Assuming you're in a heterosexual relationship, which is worse: for your partner to be unfaithful with a person of the opposite or the same sex? According to a pair of US psychologists, the answer depends on whether you're a man or woman. Men,...
- Dr. Suzanne Corkin, "gay Brain" Skeptic
Just a quick post for now on the brand new PNAS article by Savic and Lindström that spawned the "Gay Men, Straight Women Have Similar Brains" story. Professor Suzanne Corkin of MIT (famous for her studies of the amnesic patient H.M., among other things)...
- Invisible Nudes Redux
No, the invisible nudes are not taking Redux [although visible nudes might be], rather the PNAS paper on sex, sexual orientation, and implicit attentional priming by naked pix has finally been posted online: Yi Jiang, Patricia Costello, Fang Fang, Miner...
- Invisible Nudes?
From Dr Gerry Quinn: Dynamic Visual Noise You can't see them here, because the authors exploited the phenomenon of binocular rivalry to present different images to each eye separately. Specifically, dynamic visual noise and nude male or female images...
- Sweat, Urine, And Sexual Orientation
Sexual Orientation May Affect Brain Response to Human Pheromones By Miranda Hitti WebMD Medical News Reviewed By Louise Chang, MD on Monday, May 08, 2006 May 8, 2006 -- Lesbian women and heterosexual women respond differently...