How Science can be Biased

Yesterday, Psychology Today posted the article “Why are Black Women Rated Less Physically Attractive than Other Women, but Black Men are Rated Better Looking than Other Men?” by Satoshi Kanazawa. It’s easy to tear this article apart (and I will), but the bigger picture take-away message that I want to hit home is that while we all know we can mislead with statistics, we also need to know that people can mislead with science (or really, “science”).

Since the Internet is forever, even though I missed the original article, I still got the see this version here. Great, but I am unable to click any of the links, so bear with me on that technical difficulty. Oh and before you say, “la la la it’s just Psychology Today, why do you care?” I care because it perpetuates racism and besmirches the good name of evolution. I’m not the only one who cares.

Here’s the background. The author of this piece is not reporting the results of a study published in a peer-reviewed journal, but is instead reporting some analyses he did using data from Add Health, a long-term study that follows a group of people from the time they were teenagers until the present day. The study is being conducted at the University of North Carolina, and for a small fee, some of the data is available online for whoever wants it.

The point of the study was to understand how different social factors, like family relationships and education, affect health-related behaviors, like having unprotected sex. The researchers followed a “nationally representative” group of students from 1994-1995 (called Wave I) when they were in 7-12^th grade to 1996 (called Wave II) when they were in 8-12^th grade to 2001-2002 (Wave III) when they were aged 18-26. So please keep in mind that the attractiveness data presented in the Psychology Today article was taken from people aged 12 to 26.

This attractiveness data was taken by researchers after an in-home interview. Three different researchers took the data, but  even though each individual got three ratings,  at each point in time each person was only rated by one other person. The interview for Wave III takes 62 pages. Attractiveness is one question.

The author looks at attractiveness and race. The interview actually measures race in two different ways: self-reported, which allows for mixed race and specific ethnic options, and interviewer assessment of race, prompted by this: “Indicate the race of the respondent from your own observation (not from what the respondent said).” I am not sure which measure of race the author used.

So there are the limitations of the data. Now, the author says he used a factor analysis (this is a whole other kettle of fish) because they “eliminate random measurement errors.” Yes, OK, even if this was the appropriate analysis, they do not make the data better. If you are using unreliable data, such as self-reports (as in the interviewer is reporting their idea of another person’s attractiveness), the factor analysis won’t make these self-reports magically accurate. The issue isn’t with a measurement error, but with the trustworthiness of subjective (when is beauty ever objective, hm?), self-reported data.

But let’s pretend that there isn’t an inherent cultural and media bias towards “white” features and faces. Let’s pretend that these interviewers were not projecting their own prejudices and preferences, conscious or subconscious, on these teens as young as 12. Let’s pretend that the attractiveness data is completely to be trusted. Even then, the largest differences in the attractiveness of the different groups of women is no more than about 0.2 points on a 5 point scale. Statistically significant? OK. Significant enough to merit a completely factually inaccurate tirade about evolution? Not so much.

And that’s the crux of it, really. One could look at the same data and think, “Gee, there appears to be some sort of culturally-influenced preference for white features.” The interpretation of the exact same results can be wildly different depending on someone’s background. While you might believe that the research was conducted perfectly and the data and trustworthy, how that data is interpreted is still open to discussion.

Or one could look at the same data and think, “Maybe we should get a bit more data, like have ten people rate someone’s attractiveness instead of just one.”  There are very few studies that are nearly perfect, and while many scientists qualify their findings with caveats about the methods they used or the data they were able to gather, not looking critically at how the study was conducted can lead one to accept some pretty weak science. There is well-conducted science and poorly-conducted science and it is up to the consumers of that knowledge (you, me, the rest of the world) to assess for ourselves where studies fall.

Or one could look at the data and think, “Let’s see how the interviewer’s rating of attractiveness correlates to how the interviewer rated the personality of the interviewee, or how the interview rated their personal feelings of safety during the in-home visit.” What people choose to study, the questions they choose to ask, show a lot about what they find interesting and important.  The questions asked speak to the values and beliefs of the person asking the question. It would seem appropriate to ask how an interviewer’s feelings towards the interviewee might affect their assessment of attractiveness. It would seem less appropriate to ask how one person’s assessment of attractiveness can be generalized to make broad statements about the physical attractiveness of different races in relation to each other. We need to be aware that what we think of as appropriate questions can differ from person to person, even while the data stay the same.

By relying on the idea that there are significant genetic differences between the races to explain this difference in attractiveness, the author quickly brings us back to scientific racism. Calling upon a biological basis to race and using that to support white supremacy is nothing new.  One of the most famous examples, The Bell Curve, written by Richard Herrnstein and Charles Murray, claimed that environment and genetics explains differences in scores on IQ tests between races. There have been many, many vocal critics, including evolutionary biologist, Stephen J. Gould. In fact his comments on the book can be taken and directly applied to this article:

Disturbing as I find the anachronism [ie support of social Darwinism] of The Bell Curve, I am even more distressed by its pervasive disingenuousness. The authors omit facts, misuse statistical methods, and seem unwilling to admit the consequence of their own words.

Amen.