New Gaydar Study: Bogus Science?

[Important updates below!]
ResearchBlogging.org

A brief complaint published in the Edmonton Journal caught my attention when it claimed to have spotted some bogus science. [The article in the Edmonton Journal has since been taken down.] A psychology paper out of Washington University and Cornell University was published on May 17th, saying that the colloquially-famous “gaydar” may in fact be real. That is, an innate tendency for people to know whether or not someone is gay. Some people say you can tell by the way they walk, others say you’ll know by the way they talk, but this paper said that even a simple glance at a picture is enough to detect sexual orientation. Is this true?

Sexual Facial Recognition

Luckily, the entire published article is online. Researchers conducted two experiments, where they flashed a face that had been edited so as to show no hair – only their face – for 50 milliseconds (and, as you all know, there are 1000 milliseconds in a second). In the first experiment, the face is normal, and in the second experiment, the face is upside-down. Here’s what the methods section said:

The stimulus set included facial photographs of 111 gay men, 122 straight men, 87 gay women, and 93 straight women. Facial photographs were gathered from Facebook.com profiles [. . .] of individuals living in 11 major US cities who self-identified as straight or gay; photographs of self-identified bisexual people were not used as target stimuli.

So there was practically an equal number of gay and straight individuals, just as Brenda Wegmann, the person who wrote to the Edmonton Journal, had said. That alone does not cause a problem, though. The problem lies in the fact that the participants apparently knew that to begin with.

[Lead author Joshua Tabak has contacted me since the publication of this article, and has informed me that participants were not given the ratio of gay to straight people. Tabak has contacted the Edmonton Journal, and the webpage with that article has been taken down for good. My sincere apologies to the researchers.]

So participants were shown the face of a gay or straight person on a screen for only 50 milliseconds, and were asked to press one button or the other to guess sexual orientation – gay vs. straight. Do you see a problem here?

If the participants know that around half of the images they were shown are of gay people, then perhaps this would have influenced the results. What would have happened if the pictures were more representative of the actual population? What if there were only 20 pictures of self-identified gay people among a pool of 200? That’s what gaydar is, isn’t it? Because half of us aren’t gay; only a small minority of us are.

The Results are Black and White

Let’s look at the results. The graph below is taken from the paper. The lowest point (read: chance level) is 50%, topping off at 68%.

It says that people were accurate on detecting 65% of the female images shown normally and 61% upside-down; and they were accurate on 57% of the male images shown normally and 53% upside down. In other words, they had a fifty-fifty chance of getting the correct answer, and they got slightly better than chance. [This is still true, but now it is important to note that the participants were not aware that they had a fifty-fifty chance.]

But here’s another issue: You’re reading this article because it was published. With such a small number higher than chance, it is certainly conceivable that if they were to repeat the study (or add more participants) the accuracy number could drop below 50%. However, the difference in that case may be that with such flimsy and uninteresting evidence, no journal would want to publish it. [The only reason I am keeping this and the following paragraphs (on publication bias) is because they are good things to know, and they are real problems in science. I no longer think this is a major issue for this research.]

This is indeed a big problem that science has to face today – one that psychologist Dan Ariely has mentioned on the TED stage before. In fact, you could hypothetically posit that someone out there has already ran this study, but since their results were (by mere probability) below chance-levels, no journals accepted them. Journals don’t like publishing boring research because it reflects badly. This makes it seem that published articles are more substantial than they are, simply by virtue of being published.

In short, I am unimpressed by this research. [Two opinion-changing things to note: First, as I mentioned above, the participants were not given a ratio for the stimuli. But also, I erroneously reported below that only 24 participants were in the study. It turns out that there were 24 participants in experiment 1, whereas another 129 participants were included in experiment 2. The research is still not perfect, and the results are not particularly robust, but I no longer espouse that this is bogus science. It was conducted much better than I originally thought and mistakenly reported.]

The only things we can say that is informative from this research is that [Results showed that] people guessed better when the faces were female, and when they were not upside-down. Guessing higher when they’re normal vs. inverted is what we would expect to see, considering it might indicate that there are facial cues that we recognize when we see people’s faces normally. However, the difference was tiny (a measly 2% in the inversions). In fact, the results are made all the more meaningless by the fact that there were only 24 participants in the study. Twenty four, that’s it. And yet, the lead author sounds very optimistic about the results:

“It may be similar to how we don’t have to think about whether someone is a man or a woman or black or white,” said lead author Joshua Tabak, a psychology graduate student at the University of Washington. “This information confronts us in everyday life.”

Really? Is that what we’re concluding now? Sexual orientation is just as easy to visually gauge as sex and race? That offends me as a bisexual black woman. But I’m only offended to the extent that when I glance in the mirror for a split-second, I’m only 65% sure that I’m a bisexual black woman. But I guess it’s all the same, right?

No. No it is not. [In light of the edits, I admit that this is pretty strong rhetoric. But I still disagree with his statement, so I'm hesitantly letting it stand.]

Conclusion

How can this study be debunked? Very simply. Instead of flipping the faces upside-down, get Michael J Fox to draw the faces with crayons on wax paper during an earthquake, and upload those images to the same computer program. Run the same results, and you’ll still get around 50% accuracy. Don’t be surprised if you get 60% or 70% either, considering there were only 24 participants. Or, to be more scientific, you could run the experiment with 10 gay faces and 90 straight ones, and see how well participants do. [These statements were made under the impression that only 24 people participated. While they still remain true - it would certainly look very bad for this research if random guesses yielded these kinds of results - I don't expect it to be invalidated so easily anymore.]

Can this study be salvaged? The study itself isn’t a bad idea, and I applaud the researchers for trying to study this, but they evidently failed to see their methodological mistakes. First, don’t tell the participants the ratio of gay-to-straight faces. Second, reduce the number of gay pictures to make the sample of pictures more generalizable to the actual greater population. And seriously, 24 participants is an absurdly small number for a study of this scale.

[The biggest critique that I still hold is now simply the ratio of pictures, even though the participants were not told about it. The way someone should replicate the study is as follows: Run the same study, but manipulate the ratio of gay-to-straight faces as an independent variable. It would either reflect the strength of people's "gaydar," or demonstrate the limitations of this research.]

The Bottom Line

This study has not informed us of any new information. Only when and if it is replicated independently by others can we consider it with any seriousness. But the study should not be replicated as is; it should include my additions. [It's not a paradigm shift or anything, but it should certainly not be swept under the research-literature rug, as I had initially implied. It does indeed provide evidence for a facial-gaydar, but further research must discern to what extent the ratio of pictures influenced these results. For now, we can say that this research is a good start.]

So until we have a more valid procedure, we’ll have to rely on the more crude, less scientific tests of sexual orientation.

References:

Tabak JA, & Zayas V (2012). The roles of featural and configural face processing in snap judgments of sexual orientation. PloS one, 7 (5) PMID: 22629321

m4s0n501
This entry was posted in Psychology, Science, Sex and Sexuality, TED and tagged . Bookmark the permalink.

6 Responses to New Gaydar Study: Bogus Science?

  1. Pingback: Study verifies Gaydar . . . duh, no surprise | Brian David Phillips Waking Dreams

  2. Darby says:

    If the 50:50 ratio had an influence, it would seem that it would lessen any actual effect (if a participant had a “run” of 3 or 4 the same, they would tend to switch on the next whether warranted or not), so I’m not sure that’s pertinent. There’s no compelling reason to match societal demographics, since this isn’t that kind of study, and running 50:50 makes the results more understandable to more people.

    The small study group size and gender skew there seems a bigger issue, but that’s a general weakness of many of these kinds of studies.

    • Ryo says:

      It seems that you commented just before I updated this post with my edits. I apologize for that.

      But your statement on the ratio still stands. So regarding that, it’s not that I actually want it to match societal demographics; I just said that as a kind of arbitrary starting point. Maybe the 50:50 ratio makes the results understandable, but do you think the results would be the same if the ratio was 99:01, 80:20, 60:40:, etc.? I really don’t know, but it’s certainly worth further investigation. The next person to do so will probably make headlines for it regardless of the results. And if those results replicate the results of this study, that would be fantastic evidence supporting this paper.

  3. Pingback: We Got a Thing That’s Called Gaydar, Love | The SunBreak

  4. Researcher says:

    Hi,

    As bisexual, and scientist, I’ve detected a few other flaws in the research, or at least aspects that I think are in need of improvement (for further and robuster research..). Not only as a scientist but also as a member of the LGTB community.

    First, I agree with the author that the ratio of gay/straight is not a good choice, since my guess is that most participants would infer that was the actual rate despite not being told so (most of us tend to think of a 50-50 chance when intuitively asked about dichotomous choices)

    Second, the sampling method is not the most appropiate, since they’ve selected only those who identified themselves as gay, whereas most of us do not do so, neither on fb or out in many public/social contexts.. This way, their sample is likely to include more ‘visually’ gay, as they might find it more difficult (or actually not even have the chance) to hide others their sexual orientation (in a nutshell: if you act more female, you´ll be pushed earlier out of the closet, with all its further cultural implications). That could ease the process of identifying homosexual people.

    Third, they should have split the sample of ‘gay detectors’ into gay/straight, since it is my belief that being gay/bisex contributes substantially to develop your gaydar (you’re more familiar with the cultural background, clothing, styles, etc.. as well as way of looking, gestures, pose, … that help you identifying your ‘preys’.. lol).

    Fourth, although some gay people might present with some specific features (who knows why..) that makes them look ‘gay-er’ (what they call gender atypicality), the majority of us remain publicly un-open about our sexuality, and therefore unelegible as a sample for this study. As far as I’ve seen in the real world, those who look more straight (not so much atypical for your gender), as I might, are not easily recognized by anyone, which makes it easier and comfortable to keep my sexuality for me in, e.g., my office, and therefore hide it in FB and other contexts, … etc

    In the end, this and other studies, are not actually talking so much about gaydars, but about being able to identify those with atypical features for their gender, who might in turn, be gay. This could be related to hormon-related issues, or who knows… Again, for these studies to be reasonable and powerful, a big random sample (1000?) should be selected and SECRETLY asked about their sexuality (despite it, many of them would still hide it..) Then, and only then, a sample of gay/straight should try to detect their sexuality. My guessing is that we’d see how people (and more so gays) can identify those who look more female (speaking about men), but not those gays who act straight. In my personal life I’ve bumped into many gays, in the closet, who look much more straight than most of my straight friends xD

    Well, sorry if I’ve been too long, but felt like sharing my opinion with you, and who knows, maybe with the authors of the research.. I’d love having received this paper as a reviewer!! Despite its limitations, it is nice to see new studies and in this field, pushing science forward!! But I think the authors are too convincing about their conclusions despite all the limitations, and the paper’s been published in a journal with 4 points of impact factor??? That’s a pretty good level! and in my opinion, it demands a slightly more strict scientific reasoning.. but peer-review has accepted it, so must be powerful in the field. It’s my belief that pointing out the limitations of one’s own study (more so in such a polemic and difficult field) makes your research more valuable, helps publishing it and helps others to follow your topic, replicate your studies and improve everybody’s knowledge.

    Thanks for reading! ;)
    And congrats for the post and the website

    Best,

    • Ryo says:

      Thank you so much for your comment!
      You have expressed with more clarity some of the same points I was struggling to articulate, and even brought up a few that I hadn’t even thought of! Very insightful. (I hope you come back later and enlighten me on this blog again)

      I also agree that despite the limitations, it’s very nice to see that there are researchers doing interesting new research into these kinds of topics. And I hope to see more along these lines within the next few years (I was actually surprised it hadn’t really been done before!).

      Thanks again for the comment. :)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>