A brief complaint published in the Edmonton Journal caught my attention when it claimed to have spotted some bogus science. [The article in the Edmonton Journal has since been taken down.] A psychology paper out of Washington University and Cornell University was published on May 17th, saying that the colloquially-famous “gaydar” may in fact be real. That is, an innate tendency for people to know whether or not someone is gay. Some people say you can tell by the way they walk, others say you’ll know by the way they talk, but this paper said that even a simple glance at a picture is enough to detect sexual orientation. Is this true?
Sexual Facial Recognition
Luckily, the entire published article is online. Researchers conducted two experiments, where they flashed a face that had been edited so as to show no hair – only their face – for 50 milliseconds (and, as you all know, there are 1000 milliseconds in a second). In the first experiment, the face is normal, and in the second experiment, the face is upside-down. Here’s what the methods section said:
The stimulus set included facial photographs of 111 gay men, 122 straight men, 87 gay women, and 93 straight women. Facial photographs were gathered from Facebook.com profiles [. . .] of individuals living in 11 major US cities who self-identified as straight or gay; photographs of self-identified bisexual people were not used as target stimuli.
So there was practically an equal number of gay and straight individuals, just as Brenda Wegmann, the person who wrote to the Edmonton Journal, had said.
That alone does not cause a problem, though. The problem lies in the fact that the participants apparently knew that to begin with.
[Lead author Joshua Tabak has contacted me since the publication of this article, and has informed me that participants were not given the ratio of gay to straight people. Tabak has contacted the Edmonton Journal, and the webpage with that article has been taken down for good. My sincere apologies to the researchers.]
So participants were shown the face of a gay or straight person on a screen for only 50 milliseconds, and were asked to press one button or the other to guess sexual orientation – gay vs. straight. Do you see a problem here?
If the participants know that around half of the images they were shown are of gay people, then perhaps this would have influenced the results. What would have happened if the pictures were more representative of the actual population? What if there were only 20 pictures of self-identified gay people among a pool of 200? That’s what gaydar is, isn’t it? Because half of us aren’t gay; only a small minority of us are.
The Results are Black and White
Let’s look at the results. The graph below is taken from the paper. The lowest point (read: chance level) is 50%, topping off at 68%.
It says that people were accurate on detecting 65% of the female images shown normally and 61% upside-down; and they were accurate on 57% of the male images shown normally and 53% upside down. In other words, they had a fifty-fifty chance of getting the correct answer, and they got slightly better than chance. [This is still true, but now it is important to note that the participants were not aware that they had a fifty-fifty chance.]
But here’s another issue: You’re reading this article because it was published. With such a small number higher than chance, it is certainly conceivable that if they were to repeat the study (or add more participants) the accuracy number could drop below 50%. However, the difference in that case may be that with such flimsy and uninteresting evidence, no journal would want to publish it. [The only reason I am keeping this and the following paragraphs (on publication bias) is because they are good things to know, and they are real problems in science. I no longer think this is a major issue for this research.]
This is indeed a big problem that science has to face today – one that psychologist Dan Ariely has mentioned on the TED stage before. In fact, you could hypothetically posit that someone out there has already ran this study, but since their results were (by mere probability) below chance-levels, no journals accepted them. Journals don’t like publishing boring research because it reflects badly. This makes it seem that published articles are more substantial than they are, simply by virtue of being published.
In short, I am unimpressed by this research. [Two opinion-changing things to note: First, as I mentioned above, the participants were not given a ratio for the stimuli. But also, I erroneously reported below that only 24 participants were in the study. It turns out that there were 24 participants in experiment 1, whereas another 129 participants were included in experiment 2. The research is still not perfect, and the results are not particularly robust, but I no longer espouse that this is bogus science. It was conducted much better than I originally thought and mistakenly reported.] The only things we can say that is informative from this research is that [Results showed that] people guessed better when the faces were female, and when they were not upside-down. Guessing higher when they’re normal vs. inverted is what we would expect to see, considering it might indicate that there are facial cues that we recognize when we see people’s faces normally. However, the difference was tiny (a measly 2% in the inversions). In fact, the results are made all the more meaningless by the fact that there were only 24 participants in the study. Twenty four, that’s it. And yet, the lead author sounds very optimistic about the results:
“It may be similar to how we don’t have to think about whether someone is a man or a woman or black or white,” said lead author Joshua Tabak, a psychology graduate student at the University of Washington. “This information confronts us in everyday life.”
Really? Is that what we’re concluding now? Sexual orientation is just as easy to visually gauge as sex and race? That offends me as a bisexual black woman. But I’m only offended to the extent that when I glance in the mirror for a split-second, I’m only 65% sure that I’m a bisexual black woman. But I guess it’s all the same, right?
No. No it is not. [In light of the edits, I admit that this is pretty strong rhetoric. But I still disagree with his statement, so I’m hesitantly letting it stand.]
How can this study be debunked?
Very simply. Instead of flipping the faces upside-down, get Michael J Fox to draw the faces with crayons on wax paper during an earthquake, and upload those images to the same computer program. Run the same results, and you’ll still get around 50% accuracy. Don’t be surprised if you get 60% or 70% either, considering there were only 24 participants. Or, to be more scientific, you could run the experiment with 10 gay faces and 90 straight ones, and see how well participants do. [These statements were made under the impression that only 24 people participated. While they still remain true – it would certainly look very bad for this research if random guesses yielded these kinds of results – I don’t expect it to be invalidated so easily anymore.] Can this study be salvaged? The study itself isn’t a bad idea, and I applaud the researchers for trying to study this, but they evidently failed to see their methodological mistakes. First, don’t tell the participants the ratio of gay-to-straight faces. Second, reduce the number of gay pictures to make the sample of pictures more generalizable to the actual greater population. And seriously, 24 participants is an absurdly small number for a study of this scale.
[The biggest critique that I still hold is now simply the ratio of pictures, even though the participants were not told about it. The way someone should replicate the study is as follows: Run the same study, but manipulate the ratio of gay-to-straight faces as an independent variable. It would either reflect the strength of people’s “gaydar,” or demonstrate the limitations of this research.]
The Bottom Line
This study has
not informed us of any new information. Only when and if it is replicated independently by others can we consider it with any seriousness. But the study should not be replicated as is; it should include my additions. [It’s not a paradigm shift or anything, but it should certainly not be swept under the research-literature rug, as I had initially implied. It does indeed provide evidence for a facial-gaydar, but further research must discern to what extent the ratio of pictures influenced these results. For now, we can say that this research is a good start.]
So until we have a more valid procedure, we’ll have to rely on the more crude, less scientific tests of sexual orientation.
Tabak JA, & Zayas V (2012). The roles of featural and configural face processing in snap judgments of sexual orientation. PloS one, 7 (5) PMID: 22629321