Rate my professor reviews are skewed by selection bias


“He never stops smiling, he wears waistcoats, he’s awesome”

“Looks like a huge grizzly bear”

“Lectures were about as easy to follow a Leprechaun”

“Owen Wilson’s long lost brother”

These are just some of the phrases that popped up when I searched for my favorite professors on the website Rate My Professor. The ratings site allows users to post reviews of professors that they’ve taken classes with so that other students can get a sense of that specific professor’s easiness, clarity and teaching style. While the site is definitely fun, I don’t think that students should rely on the ratings system to forecast their experience in a class.

To calculate each professor’s “overall quality” rating, Rate My Professor averages scores from categories such as “helpfulness” and “clarity,” as well as others that reflect teaching style and approachability. But because posting ratings on Rate My Professor is voluntary, the data feeding into each professor’s rating is skewed by selection bias.

Selection bias is a statistical term that encompasses a variety of sampling errors that ultimately lead to a biased sample. In the case of Rate My Professor, it is likely that only students with a particularly excellent or terrible experience in class will be motivated to write a review, meaning that students who had a “just okay” or “pretty good” experience won’t be factored into the professor’s rating, even if those are the majority of students. Also, students who had a horrible experience are more likely to take action to write a review than students who had an excellent one, suggesting that some professor’s scores could be skewed for the worse.

Selection bias isn’t the only thing throwing off professor’s ratings: a study by Inside Higher Ed revealed that students are more likely to use negative words to describe female professors than male professors, and students are more likely to factor appearance into their overall opinion of female professors than male professors.

“The words ‘smart’ and ‘intellect’ are more likely to be used in ratings of men than women, and ‘genius’ is more likely to be used to describe male than female professors in all 25 disciplines for which data are available,” the study read. “… words such as ‘bossy’ are more likely to turn up in reviews of women. The same is true for ‘nurturing.’ Women are also more likely to be called ‘strict’ and ‘demanding.’”

Statistics aside, I don’t think numbers and online comments are a good way to gauge whether or not a student should take a particular course. Everyone learns differently, and therefore your experience in a certain class may vary greatly from the person next to you.

Instead of basing your decision on Rate My Professor’s Overall Quality score, talk to a friend or classmate that has a similar learning style to you. Figure out what about that professor’s teaching style worked for them and what didn’t. Better yet, email or visit the professor during office hours to chat about the class, what you’re hoping to get out of it and what the professor is planning to teach. Not only will you get a sense for whether or not you’ll click with that professor, but you’ll have a better idea of how that specific class will go.

P.S. Don’t choose classes based on Rate My Professor’s “chili pepper” feature. That’s what I would call a poor academic decision.

+ posts